Vapoursynth

[Myrsloik]

Website:
http://www.vapoursynth.com/
VapourSynth ChangeLog (https://github.com/vapoursynth/vapoursynth/blob/master/ChangeLog)
Documentation (http://www.vapoursynth.com/doc/)
List of all known plugins and scripts (http://www.vapoursynth.com/doc/pluginlist.html)
VapourSynth Editor (https://bitbucket.org/mystery_keeper/vapoursynth-editor/downloads)

[TheProfileth]

Just thought I would post the link here
http://www.vapoursynth.com
Apparently it is out and you can even install it, albeit with a python command line
http://www.vapoursynth.com/installation-instructions
Apparently you can use current avisynth plugins in it though I doubt all of them work correctly it is very interesting.
import vapoursynth
core = vapoursynth.VSCore()
core.avisynth.LoadPlugin(path='ffms2.dll')
core.avisynth.LoadPlugin(path='sangnom.dll')
a = core.avisynth.FFVideoSource (path='clouds.avi')
b = core.avisynth.SangNom (c1=a)
b.output(sys.stdout, y4m=1)
So can someone tell me exactly what this means though? Since I have just now heard of this?

[JEEB]

Apparently it is out and you can even install it, albeit with a python command line
It's still in a (relatively) closed circle testing phase, but the site is being pushed up and working. As you can see, the binary links aren't there yet, for example.

So can someone tell me exactly what this means though? Since I have just now heard of this?It means that there will soon be a (cross-platform) thing with a C API that you can plug into a scripting language of your choice (the default bindings are for Python 3.x). It also has a simplistic Avisynth plugin loading mode for Windows that should let you use most plugins as-is (MVTools and DirectShowSource seem to be some of the exceptions at the moment, but myrs is working on it).

Bottom line: Someone got some free time, and had been boiling ideas from quite a few months ago. Now they are becoming "a thing".

[TheFluff]

It's designed as a video filtering framework (sort of like Avisynth) but unlike Avisynth it:

isn't from the 90's
supports native frame-based multi-threading
is extensible (supports arbitrary per-frame metadata; new colorspaces can be defined at runtime, etc)
is cross-platform
ships with Python 3 bindings; no support for a home-grown scripting language is planned
has a C API rather than a C++ API, so it's actually portable
has native support for things like clips with changing dimensions and other properties, as well as clips with infinite duration


As JEEB said it also has an Avisynth compatibility layer that emulates the Avisynth 2.5 plugin interface; loading Avisynth plugins does not require Avisynth itself. Not all plugins are supported yet.

Doing something like this has been discussed for ages in the usual complaining-about-Avisynth circles, but now Myrsloik finally went and did it.

This is change. This is the future. This is vapour we can breathe in.
GET HYPE

[TheProfileth]

Will there be native support for higher bitdepths?

[TheFluff]

Will there be native support for higher bitdepths?

As I said, new colorspaces can be defined at runtime, so yes, there is "native support" for everything you can think up definitions for. Plugins can be written to support runtime-defined colorspaces too, since the definitions can be read programmatically.

[jmac698]

This is what I've been waiting for.

[mastrboy]

Will it be open source when released?

[TheFluff]

Will it be open source when released?

yes

edit: for the vapourware lovers, it should be noted that this thing actually exists and works already, unlike, say, avisynth 3

[Reel.Deel]

I'm assuming Vapoursynth will be in 64-bit, correct? Also, its very nice to hear that it's compatible with Avisynth plugins. Since the scripting language it's different, It's obvious Avisynth scripts won't be compatible, right?
Unless I'm missing something.

[TheFluff]

It can be built as either 32- or 64-bit, yes. No, you can't run Avisynth scripts (at least not right now); only the API is emulated, not the script parser.

[mastrboy]

What about the inbuilt avisynth functions? Are there Vapoursynth functions built-in for the stuff like: ApplyRange, SelectEven/Odd, Trim, SeparateFields... etc.
Or maybe (probably a really bad idea), a possibility to load avisynth.dll in vapoursynth?

[JEEB]

What about the inbuilt avisynth functions? Are there Vapoursynth functions built-in for the stuff like: ApplyRange, SelectEven/Odd, Trim, SeparateFields... etc.
Or maybe (probably a really bad idea), a possibility to load avisynth.dll in vapoursynth?
Yes, operations commonly used in Avisynth have already been implemented in VapourSynth. I see "SelectEvery", "Trim" and "SeparateFields" already implemented of those you mention, and quite a few others.

[clsid]

Are there any plans for a DirectShow filter that can serve as a replacement for ffdshow's AviSynth processing?

[TheFluff]

SelectEven/SelectOdd are special cases of SelectEvery and are not going to be implemented (you can of course "implement" them trivially in Python if you want). Similarly, ApplyRange is a special case of Trim (plus splicing) and can be trivially implemented in Python as well. Vapoursynth will not have an 1:1 copy of all Avisynth functions, especially not since the scripting language is now a real programming language so implementing interesting script functions is mostly trivial.

Avisynth has a ton of old crufty functions that are in the "core" for very unclear reasons, anyway. Why are there five different level manipulation functions (coloryuv, levels, limiter, rgbadjust, tweak) with a ton of overlapping functionality? Why are things like PeculiarBlend, FixBrokenChromaUpsampling and FixLuminance core functions? Why does CropBottom exist? Why does the basic "fade" function (equivalent to dissolve+blankclip) need nine different function names and prototypes? God (or maybe BenRG) alone knows.


Are there any plans for a DirectShow filter that can serve as a replacement for ffdshow's AviSynth processing?
Not at this time. It'd be pretty easy to implement for someone who is familiar with dshow though (and that's usually the bottleneck for dshow-related stuff anyway).

[mastrboy]

Good point, i only see 2 arguments that can be made for those functions,
One being It's easier to get into the script language with such "alias" functions if your not well versed in scripting, and the second one that there might be various optimizations behind the function call, like there is for mt_masktools. Like mt_average is just mt_lutxy("x y + 2 /") but faster.

[The_Paya]

I still remember me thinking "wish this was python" the first time I got to play with Avisynth.

Just Awesome.

:thanks:

For directshow sources and whatnot, here's an example of what you can do with python: https://gist.github.com/3549096
And that's just (the graph building) part of a python program I wrote to watch tv with my capture card, forcing the ffdshow filters in the graph.

[mandarinka]

I still remember me thinking "wish this was python" the first time I got to play with Avisynth.

Just Awesome.


Well, for the majority of people who know nothing about programming, avisynth not being a "proper programming language" and having all sorts of aliases like selectodd (plus dumb simple syntax) is actually much better.

I got a feeling that from the usability PoV, to us simple monkeys the glorious python thingy is going to be about as attractive as the USF subtitle format (http://blog.aegisub.org/2008/07/universal-subtitle-format-post-mortem.html) was, compared to ASS/SSA.

But who knows, maybe I just don't get it and/or hate any change.

[JEEB]

I got a feeling that from the usability PoV, to us simple monkeys the glorious python thingy is going to be about as attractive as the USF subtitle format (http://blog.aegisub.org/2008/07/universal-subtitle-format-post-mortem.html) was, compared to ASS/SSA.

You do understand that no-one actually really implemented USF writing, right? And that it really wasn't better in any real sense? It was bound to fail. Also, you just compared a scripting language with XML. IDon'tEven.

I do agree, though, that this most certainly will be less simple for "simple monkeys" until someone writes VapourSynth bindings for something more "monkey"-compatible. Or hell, someone could write a python thingy to make usage of the default VapourSynth bindings less verbose etc. The difference here is that VSynth core is really a thing, the rest is up to the people. And for those who aren't opposed to some Python, they can just use the pre-built bindings.

[TheFluff]

Monkeys have no business writing scripts any more complicated than filter().filter().filter() and that will be just as easy in python as it is in avisynth, so I don't really see your point.

[StainlessS]

If you sit a monkey at a keyboard for long enough, it would eventually compose the complete
works of TheFluff. :)

[mandarinka]

Well, I didn't mean to criticise you at all. It was actually my point that monkey (me) is fine with filter().filter().filter().

I just wanted to chill the enthusiasm a bit, because it seems to me that many people are considering this to be the "next big thing" while in fact it is a "next big thing *for function writers*"...

Users (the dudes filtering video) on the other hand generally just use linear filter chains and writing that in the python syntax is not going to be a good idea (at least it will be a lot of tiresome redundant typing).
They better not expect to be benefiting from it, else they will be disappointed. Of course, once somebody writes that avisynth script interpreter/parser, then they can use vapoursynth as backend without thinking about it (but somebody will have to write that).

[Reel.Deel]

It can be built as either 32- or 64-bit, yes. No, you can't run Avisynth scripts (at least not right now); only the API is emulated, not the script parser.

Cool. So this mean that to fully take advantage of the 64-bit perks, Avisynth plugins would have to ported to 64-bit or similar plugins would have to be developed... Right?

_____________

From a monkey's point of view, looking at some samples (http://webcache.googleusercontent.com/search?q=cache:AzWRysQG9JEJ:pastebin.com/GwtMwugn+VapourSynth&cd=4&hl=de&ct=clnk&gl=de) posted at pastebin. The scripting language doesn't *seem* that hard. But then again I'm just a monkey so I might be wrong.:p

[JEEB]

Cool. So this mean that to fully take advantage of the 64-bit perks, Avisynth plugins would have to ported to 64-bit or similar plugins would have to be developed... Right?
Yes, you would have to port stuff to make those things usable in native 64bit binary on Windows or natively on Linux (32bit/64bit). Myrsloik has shown interest in porting a plugin or two himself to native VapourSynth plugins, but in other cases people would have to work by themselves. Firesledge for example has been given access to the source code in order to both give comments and maybe port his dither package to VapourSynth :)

Also, to be honest I think that the fact that we lack something a la AvsP(mod) is more of a problem than the fact that you would have to write some more redundant stuff.

[Reel.Deel]

A 64-bit dither with a true high bit depth workflow sounds awesome. :)

So what would we use to view/render high bit depth video?

[gyth]

I just wanted to chill the enthusiasm a bit, because it seems to me that many people are considering this to be the "next big thing" while in fact it is a "next big thing *for function writers*"...
You don't have to be a function writer to use functions.

Easy access to OpenCL could give a speed boost to all the many things with pixel level parallelization.

Monkeys have no business writing scripts any more complicated than filter().filter().filter() and that will be just as easy in python as it is in avisynth, so I don't really see your point.
Can python add an implicit "last = last." to statements?

[Myrsloik]

Vapoursynth has been released. You can get it from http://www.vapoursynth.com
It is a kind of reimaging of avisynth but with python as the scripting language and many other improvements. It is still a bit incomplete and doesn't have the ability to open script files as regular video in applications... yet.

Many avisynth plugins can be loaded and used without doing anything special, this functionality is however limited to yv12 input and output. Some filters that do weird stuff simply do not work (examples of popular broken filters are mvtools2).

Report your success and failures with different avisynth filters, or the internal ones.

known issues:
absolute paths usually needed to load plugins
may deadlock with some avisynth plugins (set the number of threads, VSCore(threads=4))

r3:
fixes even more threading and performance issues
adds special support for yuy2 and rgb32 so most avisynth filters should work
added hacks to make mvtools2 work

r2:
fixes a large number of threading issues
removed the stupid clip.vi.width thing in python, now it's simply clip.width
fixed the frame reordering in the python bindings, now frames pop out in the expected order

r1:
buggy first version

[Reel.Deel]

That is a very nice surprise!! Thanks a lot for all your work. Best wishes to you and to Vapoursynth. :)


-----------------------------------------

EDIT:

For a complete python newb like me, do you mind posting an example of how to losslessly encode the output of vapoursynth.
I checked "Getting started" page but it's kinda vague.

[hajj_3]

nice, are there any speed improvements compared to avisynth so far? Maybe you could do a benchmark with a 720p x264 and 720x400 x264 and xvid with mpeg2 and h.264 sources. Does this use OpenCL acceleration yet? multithreading yet? 64bit yet?

[Myrsloik]

nice, are there any speed improvements compared to avisynth so far? Maybe you could do a benchmark with a 720p x264 and 720x400 x264 and xvid with mpeg2 and h.264 sources. Does this use OpenCL acceleration yet? multithreading yet? 64bit yet?

KNOWN ISSUES IN R1: I messed up the python bindings slightly so you may get frames out of order when you pipe stuff. TDecimate is also derp. You've been warned. R2 coming in 36h or so.

It does use frame level multithreading but I haven't done a lot of speed tests yet but it should scale fairly well though.

And a few words about 64bits. Good but not magic. You get most of the speed benefits in 32bit code by using sse2 instructions so the gap isn't huge. The second benefit with address space is something I hope to simply get around with better memory management. X64 will come but I think most people will value partial avisynth binary compatibility higher.

Gpu stuff is somewhere in the slightly distant future. I need a stable base first.

[Chikuzen]

>>> import vapoursynth as vs
>>> c = vs.Core()
>>> c.avs.LoadPlugin(path=r'C:\AviSynth\plugins_x86\rawsource26.dll')
>>> clip = c.avs.RawSource(file=r'D:\video\derf\soccer_4cif.y4m')
>>> print(clip.videoinfo.width)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'vapoursynth.VideoNode' object has no attribute 'videoinfo'
>>> print(clip.vi.width)
704


almost all users don't know the variable name of the struct VideoInfo defined by source filter plugin.
need function GetVideoInfo().

[naoan]

Well, I didn't mean to criticise you at all. It was actually my point that monkey (me) is fine with filter().filter().filter().

I just wanted to chill the enthusiasm a bit, because it seems to me that many people are considering this to be the "next big thing" while in fact it is a "next big thing *for function writers*"...

Users (the dudes filtering video) on the other hand generally just use linear filter chains and writing that in the python syntax is not going to be a good idea (at least it will be a lot of tiresome redundant typing).
They better not expect to be benefiting from it, else they will be disappointed. Of course, once somebody writes that avisynth script interpreter/parser, then they can use vapoursynth as backend without thinking about it (but somebody will have to write that).

How about getting cooler and better scripts from the developers because it's gotten easier to write and maintain script now?

I'd say that more power to the devs, the more power to end users too, no? :rolleyes:

[Myrsloik]

I've posted the second version now. It should be stable enough for general experimentation now.

[TheFluff]

For those who aren't feeling the hype, it might be worth mentioning that this release can multithread many Avisynth filter chains better than Avs-MT can.

[Myrsloik]

For those who accept numbers pulled out of dark body cavities, here are some examples from yesterday evening:
FFVideoSource().4xTnlMeans() (maybe not the most appealing filter chain)
VS: 35s
Avisynth: 130s

FFVideoSource().4xTFM() (I know, retarded)
VS: 30s
Avisynth: 50s

This was on a core i5 3570k. There are probably also a few more improvements that can be made to thraed scheduling/caching in later releases. There's also a vfw module coming some day in the not too distant future (open python scripts as video just like .avs files)

[mastrboy]

Is there a way to load vapoursynth scripts in virtualdub or avspmod for previewing? Currently i have only managed to output raw yv12 video data on the command line...

[Myrsloik]

Is there a way to load vapoursynth scripts in virtualdub or avspmod for previewing? Currently i have only managed to output raw yv12 video data on the command line...

No, not yet. The next release will most likely have some kind of support for it.

[TheFluff]

Is there a way to load vapoursynth scripts in virtualdub or avspmod for previewing? Currently i have only managed to output raw yv12 video data on the command line...

The VfW module Myrsloik is talking about will let you do that. Currently the only way is to pipe the output to x264 or some other compression tool that supports y4m input.

[Chikuzen]

I tried the following.

#!/bin/env python3
# vs_test1.py
import sys
import vapoursynth as vs
core = vs.Core()
core.avs.LoadPlugin("rawsource26.dll")
clip = core.avs.RawSource(file="soccer.y4m")
clip.output(sys.stdout, y4m=True)

> python vs_test.py > out.y4m

However, this script only outputted y4m header, and it did not work after it at all.

It seems that vapoursynth dislikes avisynth source filters other than ffms2 :(

[Myrsloik]

I tried the following.

#!/bin/env python3
# vs_test1.py
import sys
import vapoursynth as vs
core = vs.Core()
core.avs.LoadPlugin("rawsource26.dll")
clip = core.avs.RawSource(file="soccer.y4m")
clip.output(sys.stdout, y4m=True)

> python vs_test.py > out.y4m

However, this script only outputted y4m header, and it did not work after it at all.

It seems that vapoursynth dislikes avisynth source filters other than ffms2 :(
http://forum.doom9.org/showpost.php?p=1589319&postcount=142

Only avisynth 2.5 plugins supported.

[kolak]

Will it be possible to do something like virtual file system?

http://forum.doom9.org/showthread.php?t=133313

[active1]

thanx! i'll try it now :)
stupid question: what is the advantage of VapourSynth over Avisynth MT?

[Myrsloik]

Will it be possible to do something like virtual file system?

http://forum.doom9.org/showthread.php?t=133313

I suppose so. You could probably reuse quite a bit of the sourcecode so replacing avisynth as the backend should be quick. Or so my 15 second survey of the sourcecode tells me.

thanx! i'll try it now :)
stupid question: what is the advantage of VapourSynth over Avisynth MT?

VapourSynth may one day have all its threading issues fixed. I'm not so optimistic about avisynth mt...

And with that I return to my debugger.

[Rumbah]

The site seems to be down, is there another way to download it?

[sneaker_ger]

http://www.mediafire.com/?64s562ufwy8x7fp (Windows r2)

[Myrsloik]

The site seems to be down, is there another way to download it?

Not for the moment, no. I will make a third release once it's back up. I guess you get what you pay for... and I pay $1.78/month.

Anyway, have some performance comparisons from r3 while you wait:
Avisynth 2.5.8
$ time avs2yuv.exe test.avs - > NUL
test.avs: 640x368, 24000/1001 fps, 5000 frames

real 1m23.680s
user 0m0.000s
sys 0m0.015s

VapourSynth r3:
$ time /c/Python32/python.exe test.py > NUL

real 0m23.753s
user 0m0.015s
sys 0m0.000s
Script:

import vapoursynth as vs
import sys
core = vs.Core(threads=4)
core.avs.LoadPlugin(path='ffms2.dll')
core.avs.LoadPlugin(path='mvtools2.dll')
ret = core.avs.FFVideoSource(source='possible rule 6 violation.avi', threads=0)
ret = core.std.Trim(clip=ret, first=10000, length=5000)

src = ret
c = core.avs.MSuper(c1=src)
b1v = core.avs.MAnalyse(c1=c, delta=1, isb=False)
f1v = core.avs.MAnalyse(c1=c, delta=1, isb=True)
b2v = core.avs.MAnalyse(c1=c, delta=2, isb=False)
f2v = core.avs.MAnalyse(c1=c, delta=2, isb=True)
ret = core.avs.MDegrain2(c1=src, c2=c, c3=b1v, c4=f1v, c5=b2v, c6=f2v)
ret.output(sys.stdout, y4m=True)

[active1]

wow, vapoursynth is really awesome!
thank you Myrsloik :)
Q: can i load avsi scripts with it?

I'm not so optimistic about avisynth mt...

can you explain 'why' a little?

[hajj_3]

can you explain 'why' a little?

because there has been virtually no development on avisynth for ages. Vapoursynth is already good and judging by the speed of changes i'm sure we'll have a great version ready within a few months with lots of added features.

[Myrsloik]

The website has stopped sleeping. R3 has been released. It has greatly improved avisynth plugin compatibility and many other fixes. Most notably mvtools works perfectly now so someone crazy could try porting the QTGMC script.

Next on the list is probably adding more things related to frame properties so they can become useful, including a port of ffms2.

[-Vit-]

I've been thinking about the QTGMC port, if I get some time this weekend I'll have a go.

[Keiyakusha]

Not working...
http://i.imgur.com/SVPa1.png

[Myrsloik]

Not working...


\ is the escape character for python strings, just like in C.
Either write path=r'c:\blah.dll', path='c:/blah.dll' or path='c:\\blah.dll' to make it work as expected.

[Keiyakusha]

Oh yes, this way it works... Sorry.

[-Vit-]

I get problems encoding through y4m to x264 with core threads at anything other than 1. Affects any source. Can post scripts if you wish, but it's just a pass-through python script (FFVideoSource) & generic x264 settings. Get errors like this and encode terminates early (usually after a few frames, but a non-deterministic #):
y4m [error]: bad header magic (82848485 <=> àääéé)
y4m [error]: bad header magic (41524682 <=> éFRAM)

Thread count on FFVideoSource works fine. No problems so far experimenting with 1 core thread.

[JEEB]

I get problems encoding through y4m to x264 with core threads at anything other than 1. Affects any source. Can post scripts if you wish, but it's just a pass-through python script (FFVideoSource) & generic x264 settings. Get errors like this and encode terminates early (usually after a few frames, but a non-deterministic #):
y4m [error]: bad header magic (82848485 <=> àääéé)
y4m [error]: bad header magic (41524682 <=> éFRAM)

Seems to be related to how x264 reads the stuff, if output it to a y4m file or switch x264 to avconv/ffmpeg, it will work :s
(Chikuzen had this problem last night)

[-Vit-]

Same problem with ffmpeg. Fine with core threads = 1. If core threads is anything else then it encodes only a few frames (final frames become offset to the right), no error message shown in ffmpeg. Seems like a threading issue to me.

[JEEB]

Same problem with ffmpeg. Fine with core threads = 1. If core threads is anything else then it encodes only a few frames (final frames become offset to the right), no error message shown in ffmpeg. Seems like a threading issue to me.
Yeah, in that case it's a threading/output-related issue. At least Myrs is already debugging it.

[wOxxOm]

What is the easiest way of making vapoursynth case-insensitive for filter and parameter names (I mostly write everything in lowercase) ?

[TheFluff]

What is the easiest way of making vapoursynth case-insensitive for filter and parameter names (I mostly write everything in lowercase)?

Python function and variable names are always case sensitive; that's a language thing and not something VS gets to decide. Probably the simplest way of making it possible to write everything in lowercase would be write some metaprogramming magic that generates all-lowercase wrappers for all the methods on the VS core object.

[wOxxOm]

What are 'format' and 'yuvrange' and is it possible to mimick spline16/36/64resize with Spline(clip:clip;width:int:opt;height:int:opt;format:int:opt;yuvrange:int:opt;) ? Is it possible to specify float numbers for the area to be resized like in avisynth's resize filters which have cropXXXX parameters? What is the syntax of 'Lut' and 'Lut2' functions? Is it possible to implement positional parameters so that instead of Spline(c1=myvideonode,width=1280,height=720) it'd be spline(myvideonode,1280,720) ? It would be nice to have a callback in 'output' function with current frame number and coded frame size so that it could be used in a custom gui for progress reporting

[active1]

will be there any script editor like AvsP(mod) to view source, editing, etc ?

[wOxxOm]

There are examples of implementing a case-insensitive dictionary which might work for method names as well if I am not mistaken (here (http://stackoverflow.com/questions/2082152/case-insensitive-dictionary) and there (http://code.activestate.com/recipes/66315)):class CaseInsensitiveDict(dict):
def __setitem__(self, key, value):
super(CaseInsensitiveDict, self).__setitem__(key.lower(), value)

def __getitem__(self, key):
return super(CaseInsensitiveDict, self).__getitem__(key.lower())Other methods should also be redefined in this fashion if being used, like 'contains', 'has_key', etc.

It looks quite easy to incorporate into vapoursynth's avs class, probably even as an optional parameter 'casesens' in vapoursynth.Core() method which would instantiate avs class or avsCaseInsensitive, yet I don't know if and how I can override the avs class from inside my code...

[Myrsloik]

There are examples of implementing a case-insensitive dictionary which might work for method names as well if I am not mistaken (here (http://stackoverflow.com/questions/2082152/case-insensitive-dictionary) and there (http://code.activestate.com/recipes/66315)):class CaseInsensitiveDict(dict):
def __setitem__(self, key, value):
super(CaseInsensitiveDict, self).__setitem__(key.lower(), value)

def __getitem__(self, key):
return super(CaseInsensitiveDict, self).__getitem__(key.lower())Other methods should also be redefined in this fashion if being used, like 'contains', 'has_key', etc.

It looks quite easy to incorporate into vapoursynth's avs class, probably even as an optional parameter 'casesens' in vapoursynth.Core() method which would instantiate avs class or avsCaseInsensitive, yet I don't know if and how I can override the avs class from inside my code...

I suppose I could allow lower case versions too since it's at least in the spirit of python. The uppercasing has bothered me too since python prefers_this_style. Maybe I'll just change the bindings to do everything as lower case.

[Myrsloik]

What are 'format' and 'yuvrange' and is it possible to mimick spline16/36/64resize with Spline(clip:clip;width:int:opt;height:int:opt;format:int:opt;yuvrange:int:opt;) ? Is it possible to specify float numbers for the area to be resized like in avisynth's resize filters which have cropXXXX parameters? What is the syntax of 'Lut' and 'Lut2' functions? Is it possible to implement positional parameters so that instead of Spline(c1=myvideonode,width=1280,height=720) it'd be spline(myvideonode,1280,720) ? It would be nice to have a callback in 'output' function with current frame number and coded frame size so that it could be used in a custom gui for progress reporting

1. yuvrange isn't implemented yet in the filter, I'll add it so it works as good/bad as swscale in libav does. I'll go and ask how much is implemented now, last time I looked it was only half done. I have no idea about replicating the avisynth spline functions. Tell me if it looks similar to any of them.
2. Not sure if swscale implements it, will go ask the developers because I'm curious too.
3. For Lut the lut argument is a list of 2^(video bits per sample) ints. So for normal video it simply is the 256 values in the lut you've calculated by yourself. Lut2 is similar except that it's Lut2(clips=[clip1, clip2], lut=[2^(usually 16) ints])
You can also add planes=[1, 2] (or similar) if you only want it to work on uv in yuv.
4. I like it verbose, maybe some day I'll relax the requirement but I see it as a good way of keeping scripts readable.
5. It shall be added

[Guest]

There are two threads going on Vapoursynth in Avisynth Develoment now. Is there any reason not to merge them?

[JEEB]

There are two threads going on Vapoursynth in Avisynth Develoment now. Is there any reason not to merge them?
I'm not Myrs, but the "Vapoursynth" thread might as well be merged here, that way there'd be only one thread + Myrs can update the top post.

[TheProfileth]

Yes, I agree they should be merged also.

[Myrsloik]

The users have spoken. Make it so.

[Guest]

I have waved my magic wand and made it so.

Now you try to figure out how I gave you a first thread post like I did, which you can fill in as needed. :eek:

[Myrsloik]

R4 posted at the website. It should fix the corrupt frame output and some other stuff too.

[TheRyuu]

ffms2-r712.7z (https://ffmpegsource.googlecode.com/files/ffms2-r712.7z)

As seen in the ffmpegsource thread, contains vapoursynth support.

[CarlPig]

I can't get it to work.

http://cloudho.st/i/P1XneLpC.jpg

[06_taro]

python is case sensitive, the function is "Core()", not "core()"

[Yoshiyuki Blade]

That's further than what I got. I can't even get past the first line (says specified module no found) :eek:! This is kind of frustrating because the instructions really are straightforward and easy to follow. Everything is (presumably) at default settings too.

[TurboPascal7]

Awesome project. Finally, something that can completely replace avisynth one day.

I played around with it and wrote a simple (buggy, ugly etc.) wrapper to provide more avisynth-like syntax (chaining support, positional arguments, case insensitivity): http://privatepaste.com/0b53562cdb
While it definitely shouldn't be used for anything real in its current state, I hope it shows that vapoursynth could benefit from better API.

Thoughts:
1. list_functions() should return a string and not output it to stdout by itself. IMHO, this is just wrong (see _rebuild_functions_cache function).
2. There should be a list_functions() analog, that would return the same data in dictionary/list/whatever form. I'm pretty sure you have one internally. Possible use case: editors, wrappers.
3. Chaining support, or you can call it 'fluent interface'. Just because it makes things so much easier.
4. Positional argument. I'm not sure why you like it verbose while most programming languages use this kind of args without any problems. Some don't even support named arguments.
5. Case-insensitivity should be added. Or at least lowercase bindings for all functions. Good editor could help, sure, but right now it takes me ages to type FFVideoSource correctly.
6. Some list_functions() analog for core functions. Because your (cython?) objects don't support __dict__, inspect.getargspec and other reflection methods. This can be solved with good documentation too. :)

I'm not very good with python and I'm pretty sure that some of these questions can be solved by a simple correct wrapper without your help. Having all this in core API would be nice though.

Also, optional parameters don't seem to work right now. And CropAbs accepts negative width/height but then outputs garbage. IMHO it should behave like avisynth's crop function.

[ajp_anton]

So what are the long-term plans for this? To be able to do everything Avisynth can do, but better*?

*Working multi-threading, arbitrary bit-depths and color spaces.

[Myrsloik]

So what are the long-term plans for this? To be able to do everything Avisynth can do, but better*?

*Working multi-threading, arbitrary bit-depths and color spaces.

To crush all who stand against me! *muahaha*

[active1]

hi, i can't make CPU Usage = 99% when i use one filter
the CPU Usage is 25% all the time, but when i use the filter x4, the CPU Usage become 99%
but in avisynth MT the CPU Usage is 99% all the time
what is the problem ?? (or is it a problem ??)

[Myrsloik]

Awesome project. Finally, something that can completely replace avisynth one day.

I played around with it and wrote a simple (buggy, ugly etc.) wrapper to provide more avisynth-like syntax (chaining support, positional arguments, case insensitivity): http://privatepaste.com/0b53562cdb
While it definitely shouldn't be used for anything real in its current state, I hope it shows that vapoursynth could benefit from better API.

Thoughts:
1. list_functions() should return a string and not output it to stdout by itself. IMHO, this is just wrong (see _rebuild_functions_cache function).
2. There should be a list_functions() analog, that would return the same data in dictionary/list/whatever form. I'm pretty sure you have one internally. Possible use case: editors, wrappers.
3. Chaining support, or you can call it 'fluent interface'. Just because it makes things so much easier.
4. Positional argument. I'm not sure why you like it verbose while most programming languages use this kind of args without any problems. Some don't even support named arguments.
5. Case-insensitivity should be added. Or at least lowercase bindings for all functions. Good editor could help, sure, but right now it takes me ages to type FFVideoSource correctly.
6. Some list_functions() analog for core functions. Because your (cython?) objects don't support __dict__, inspect.getargspec and other reflection methods. This can be solved with good documentation too. :)

I'm not very good with python and I'm pretty sure that some of these questions can be solved by a simple correct wrapper without your help. Having all this in core API would be nice though.

Also, optional parameters don't seem to work right now. And CropAbs accepts negative width/height but then outputs garbage. IMHO it should behave like avisynth's crop function.

1. will be changed for the next version
2. will probably happen some day
3. suggest a way to implement it that's not completely unpythonian and I'll consider it
4. some day
5. I will not add case insensitive stuff to a case sensitive language. Maybe some day I'll make lowercase versions acceptable too.
6. It's already included in list_functions(), why do you need one more?

I think I've fixed the crop argument checks. Can you give an example of the optional arguments maybe not working?

[Myrsloik]

hi, i can't make CPU Usage = 99% when i use one filter
the CPU Usage is 25% all the time, but when i use the filter x4, the CPU Usage become 99%
but in avisynth MT the CPU Usage is 99% all the time
what is the problem ?? (or is it a problem ??)

It is how my threading model works. Avisynth filters are for compatibility reasons limited so one instance may run on one core at one time. Avisynth-mt gets around this by creating multiple instances (one per thread), OR by making very unsafe assumptions.
Newly written and ported filters for the vs api can if done correctly handle multiple frames at once. See the included Lut/Lut2 function.

[active1]

It is how my threading model works. Avisynth filters are for compatibility reasons limited so one instance may run on one core at one time. Avisynth-mt gets around this by creating multiple instances (one per thread), OR by making very unsafe assumptions.
Newly written and ported filters for the vs api can if done correctly handle multiple frames at once. See the included Lut/Lut2 function.

so there is a chance to make it even better than Avisynth MT in speed wise? (with only one filter)

[TurboPascal7]

I think I've fixed the crop argument checks. Can you give an example of the optional arguments maybe not working?

Traceback (most recent call last):
File "D:/test.py", line 21, in <module>
source = w.std.CropAbs(clip=source, x=16, y=16)
File "vapoursynth.pyx", line 434, in vapoursynth.Function.__call__ (cython\vapoursynth.c:7371)
vapoursynth.Error: 'CropAbs: argument width is required'

source = w.std.CropAbs(clip=source, x=16, y=16, width=-16, height=-16)
Works but outputs garbage. Should throw error or work like avisynth's crop (subtracting provided negative width/height values from clip's parameters).

6. It's already included in list_functions(), why do you need one more?
I don't really need another list_functions() method. I do want to know Core class constructor parameters, VideoNode.output() method parameters (wanted to test callback, added in r4) etc. Without documentation and any public API I don't see any ways to do it since reflection doesn't work well with cython. I'm asking for this too early, I guess.

[Myrsloik]

so there is a chance to make it even better than Avisynth MT in speed wise? (with only one filter)

Not without porting filters, no. I'm not going to make another avisynth-mt mess. I intend to win people over in the end by being faster and more stable for most users.

[Myrsloik]

Traceback (most recent call last):
File "D:/test.py", line 21, in <module>
source = w.std.CropAbs(clip=source, x=16, y=16)
File "vapoursynth.pyx", line 434, in vapoursynth.Function.__call__ (cython\vapoursynth.c:7371)
vapoursynth.Error: 'CropAbs: argument width is required'

Of course width is required, this only moves the upper left point.

source = w.std.CropAbs(clip=source, x=16, y=16, width=-16, height=-16)
Works but outputs garbage. Should throw error or work like avisynth's crop (subtracting provided negative width/height values from clip's parameters).

fixed


I don't really need another list_functions() method. I do want to know Core class constructor parameters, VideoNode.output() method parameters (wanted to test callback, added in r4) etc. Without documentation and any public API I don't see any ways to do it since reflection doesn't work well with cython. I'm asking for this too early, I guess.

Core constructor: def __cinit__(self, flatten = True, addcache = True, int threads = 0)

Not much of interest, really. Flatten is whether or not single return value functions get the returned dict flattened to a single value automatically. addcache controls automatic cache insertion.

Output declaration:
def output(self, object fileobj not None, bint y4m = False, int lookahead = 10, object progress_update = None):

Where lookahead is the number of frame requests it has going at once to keep throughput up and progress_update is a functions that could looks like this:
def print_frame(n, total):
print('Frame: %d/%d' % (n, total))

[active1]

Not without porting filters, no. I'm not going to make another avisynth-mt mess. I intend to win people over in the end by being faster and more stable for most users.

that's good too, i hope that we will not need to load any filter from avisynth someday.

i have a problem with ColorMatrix:
http://data.imagup.com/10/1161861426.jpg
every filter above work very well, except ColorMatrix
is it not supported yet?

[Myrsloik]

I released R5. Changes in the first post. If this one turns out well I'll start experimenting with other platforms.

[TheProfileth]

r5:
more python output fixes
added accept_lowercase as a setting for the core constructor
python module refinement, now all objects have a string representation with more information
other minor fixes
Does this mean you can do things with all lowercase now?

[Myrsloik]

Does this mean you can do things with all lowercase now?

Yes, I added it as an experiment and it's off by default. It may disappear again if I don't like it.

[wOxxOm]

this is great, let it be :-)
also, is there a way yet to create arbitrary clips like in avisynth's ScriptClip? The only way I see now is to loop through all frames and then core.std.splice individual frames after some processing which seems to defy the idea of lazy calculation in a ScriptClip.

[Reel.Deel]

I have managed to finally output y4m but I do not know how to successfully pipe Vapoursynth to X264. Can someone please post a sample of how to achieve this?

@ Myrsloik

So I noticed that the vsvfw.dll is no longer included in r5. Is it still needed?

[Myrsloik]

I have managed to finally output y4m but I do not know how to successfully pipe Vapoursynth to X264. Can someone please post a sample of how to achieve this?

@ Myrsloik

So I noticed that the vsvfw.dll is no longer included in r5. Is it still needed?

It was never used. The vfw part still isn't anywhere near ready.

[TheFluff]

that's good too, i hope that we will not need to load any filter from avisynth someday.

i have a problem with ColorMatrix:
every filter above work very well, except ColorMatrix
is it not supported yet?

It's not supported yet. Vapoursynth doesn't have equivalents for all of the internal Avisynth functions yet, so filters like ColorMatrix that invoke internal Avisynth filters will break if the invoked function doesn't exist in Vapoursynth.

[Myrsloik]

It's not supported yet. Vapoursynth doesn't have equivalents for all of the internal Avisynth functions yet, so filters like ColorMatrix that invoke internal Avisynth filters will break if the invoked function doesn't exist in Vapoursynth.
I'd like to add that I may never add any extensive invoke support on the avisynth side. I estimate that the time to port the few useful filters that use invoke and adding good enough support for it in vs is about the same. I know which one I'm going to pick.

[Yoshiyuki Blade]

Ok so I finally got past the "test" phase of the installation instructions LOL. It turns out that I needed the Microsoft Visual C++ 2010 Redistributable Package (http://www.microsoft.com/en-us/download/details.aspx?id=5555) (instead of the 2008 one) to get it working. The list.functions() is messed up in r5 compared to r3 (I don't have r4 around for some reason). Can't wait to dabble with a new interface and learn new things.

[smok3]

vapour is win only?

[TurboPascal7]

The list.functions() is messed up in r5 compared to r3 (I don't have r4 around for some reason).
It isn't messed up. It just returns a string instead of printing it. Should be used like print(core.list_functions()).

[Yoshiyuki Blade]

It isn't messed up. It just returns a string instead of printing it. Should be used like print(core.list_functions()).

I see. Yeah, it looks fine when used that way.

[Myrsloik]

vapour is win only?

Only until it's kinda stable.

[Reel.Deel]

When I try loading the "official" MVtools (v2.5.11.3) it works fine, but when I try it on Dithers MVTools (v2.6.0.5) it just silently crashes the python command line.

Also, I'm happy to see DGIndexNV work on Vapoursynth. :)

[Chikuzen]

Also, I'm happy to see DGIndexNV work on Vapoursynth. :)

However, DGMPEGDec can't be used because it requires avs internal crop function.
I used TS2AVI/MPEG2Dec3k after a long time...

[Guest]

However, DGMPEGDec can't be used because it requires avs internal crop function.
I can make a version without internal cropping if you like.

[Reel.Deel]

I can make a version without internal cropping if you like.

Please do. That would be very nice. Thanks neuron2!

[wOxxOm]

Regarding updating of avisynth plugins: as far as I can see, it's possible to make them native vapoursynth plugins by providing _VapourSynthPluginInit in addition to _AvisynthPluginInit2, so what are the benefits in general?

[Keiyakusha]

Also, I'm happy to see DGIndexNV work on Vapoursynth. :)
I don't understand why you need DGIndexNV if you have FFMS2 (which is free and awesome). if you have not very ancient CPU, nvidia decoding is not faster at all and not safe with more than one instance (or if you use some memory eating gpu filters). What are other possible benefits?

However, DGMPEGDec can't be used because it requires avs internal crop function.
I used TS2AVI/MPEG2Dec3k after a long time...

When and what for if uses crop? You mean DgdecNV when decoding 1080p blu-ray? It should be possible to disable crop so it will output 1920x1088.
On the other hand DGMPEGDec Is what I'd like to see too. Not that it is irreplaceable, I'm just so used to it...

[TheFluff]

Regarding updating of avisynth plugins: as far as I can see, it's possible to make them native vapoursynth plugins by providing _VapourSynthPluginInit in addition to _AvisynthPluginInit2, so what are the benefits in general?

Native VS plugins can:
- interact directly with the VS thread pool (= potentially better multithreading performance, may require some work on the plugin though)
- support high bitdepth colorspaces (and some other exotics such as planar RGB) natively
- store arbitrary per-frame metadata (such as VFR timestamps) without having to resort to hiding it in obscure ways (the decomb/tfm hinting information is one example of plugins doing that)

[mandarinka]

Isn't FFMS2 (because of limitations in libavcodec) unable to properly decode interlaced h.264 streams? (At least when muxed in TS.) IIRC it also had problems with VC-1 streams.

[Myrsloik]

Regarding updating of avisynth plugins: as far as I can see, it's possible to make them native vapoursynth plugins by providing _VapourSynthPluginInit in addition to _AvisynthPluginInit2, so what are the benefits in general?

Unless you're willing to do a bit of extra coding, or at least code auditing, none. Here are some of my observations from looking at filter code about what different things you could expect:
1. Several filters use planar YUV422 internally and have to pack and unpack. These could quite easily be changed to directly accept planar YUV for a small speedup for YUY2.
2. Many of the same filters in category one are quite general and probably could be extended to full resolution planar YUV support at least with a bit more effort.
3. Native filters can select their best threading model. For example a filter like ColorMatrix probably never updates its conversion tables after initialization so they're safe to share. This means that the filter can be run on several frames in parallel.
4. If you lay off the inline assembler (you can still use yasm+that magic x264 asm.inc file) your plugins will work and compile on every civilized desktop OS from a single codebase.
5. No more hacks.

[Reel.Deel]

I don't understand why you need DGIndexNV if you have FFMS2. if you have not very ancient CPU, nvidia decoding is not faster at all and not safe with more than one instance. What are other possible benefits?

I was just trying it out to see if it worked. Also IIRC FFMS2 was not frame accurate with VC-1 and and did not like interlaced H.264 (don't know if that's still the case).
Futhermore, I have a handfull of blu-rays that the movie is cut up into a 100+ segments and it's very easy just to drag the mpls onto DGIndexNV and index the entire movie. BTW I do have an ancient CPU (not for long) :).

[Keiyakusha]

it's very easy just to drag the mpls onto DGIndexNV and index the entire movie.
Hmm, didn't know that. I had impression that it only can work with single segment at one time or something like that. Maybe will try it someday.(I, as many others I guess, just remux the movie first, to make it one) Interlaced streams maybe true. I never saw any interlaced bluray though (or tv broadcasts that is not mpeg2)

[hajj_3]

uk tv show blurays are 1080i for uk created tv shows.

[Guest]

Also remember that DGDecNV can use the GPU for deinterlacing/resizing and that can bring a substantial speedup. Sure it's not QTGMC but the quality is pretty good and so offers a useful performance point. DGDecNV can load multiple files (including automatically from an MPLS playlist file); it's the old DGAVCDec that could open only one file.

Regarding DGMPGDec, it contains an internal Invoke call, which I assume is the source of the problem with Vapoursynth. I'd like to understand the problem better though. Is it only an issue for 1088 streams that get cropped automatically to 1080? Obviously we can't enable cropping in the GUI as that will trigger an Invoke("crop") in DGDecode, but if we have a non-1088 stream with no cropping enabled in the GUI does everything work?

[Reel.Deel]

I tried DGMPGDec with a true 1080 stream (cropping disabled) and it gave me the following error:
Invoke not fully implemented, tried to call: crop but I will pretend it doesn'texist
Avisynth Error: escaped IScriptEnvironment::NotFound exceptions are non-recoverable, crashing...

BTW the file I used was 1080 not 1088.

*edit*

Just to be sure I tried DGMPGDec (same settings as before) with both a NTSC DVD stream and a 720p60 stream and they both worked. :)

[Guest]

Yes, I know about the automatic 1088->1080 issue!

I'm asking if it fails when we don't have that case. Try an SD stream for example, with no cropping enabled in the GUI.

[tebasuna51]

Just yesterday I make some test with r3.

With r5 (I never can download r4):

Python 3.2.3 (default, Apr 11 2012, 07:15:24) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import vapoursynth as vs
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: DLL load failed: No se puede encontrar el módulo especificado.

XP SP3 without changes from yesterday.
Reload r3 and work fine.

[Reel.Deel]

Had the same problem yesterday. Try re-downloading r5 again and re-replace the binaries. It worked for me.

[Myrsloik]

Just yesterday I make some test with r3.

With r5 (I never can download r4):



XP SP3 without changes from yesterday.
Reload r3 and work fine.

Did you download it within one hour of release? I forgot to select static linking and silently updated it.
If not I have no idea why you're getting that error since no dependencies have changed.

[Guest]

BTW the file I used was 1080 not 1088. No, it's coded as 1088. DGIndex hides that from you. It it were not coded as 1088, then the Invoke() call would not be made. If you are still adamant that it is 1080, then please post a sample.

Just to be sure I tried DGMPGDec (same settings as before) with both a NTSC DVD stream and a 720p60 stream and they both worked. :) OK, sweet, thank you.

[tebasuna51]

Did you download it within one hour of release? I forgot to select static linking and silently updated it.

Thanks, now work r5.

[06_taro]

Did some tests, and *.avs.MPEG2Source(d2v=r"d2vpath") works for SD sources.

BTW, I encountered "QWaitCondition: Destroyed while threads are still waiting" when finishing a session if I use Core() instead of Core(threads=1). Only occurs in r5.

[Myrsloik]

Did some tests, and *.avs.MPEG2Source(d2v=r"d2vpath") works for SD sources.

BTW, I encountered "QWaitCondition: Destroyed while threads are still waiting" when finishing a session if I use Core() instead of Core(threads=1). Only occurs in r5.

That warning is harmless. It's just that I haven't made the destruction of the threadpool completely correct.

[wOxxOm]

ok, I've ported (http://www.privatepaste.com/db2acec810) avisynth's YAHR (http://avisynth.org/mediawiki/YAHR) to vapoursynth, apparently p lame as it uses only 30% cpu (20% in yahrmask), so what are the guidelines to paralellize spatial processing with these widely used avisynth plugins in vapoursynth/python? Or probably the more actual question is how to parallelize it when there are temporal filters like tdecimate?

[Myrsloik]

ok, I've ported (http://pastebin.com/vuHjKtaK) avisynth's YAHR (http://avisynth.org/mediawiki/YAHR) to vapoursynth, apparently p lame as it uses only 30% cpu (20% in yahrmask), so what are the guidelines to paralellize spatial processing with these widely used avisynth plugins in vapoursynth/python? Or probably the more actual question is how to parallelize it when there are temporal filters like tdecimate?

Do you get any warnings about frames not being prefetched? And can you share your converted script and the exact plugisn you used an I will take a look at it.

[Guest]

Invoke not fully implemented, tried to call: crop but I will pretend it doesn'texist
Avisynth Error: escaped IScriptEnvironment::NotFound exceptions are non-recoverable, crashing... Does this mean that Vapoursynth took an exception or DGDecode took an exception? Does Vapoursynth return an error for my Invoke call? It would be nice if it could simply be ignored. I'm not sure though who should do the ignoring. Thank you.

[Myrsloik]

Does this mean that Vapoursynth took an exception or DGDecode took an exception? Does Vapoursynth return an error for my Invoke call? It would be nice if it could simply be ignored. I'm not sure though who should do the ignoring. Thank you.

All crop calls will simply be ignored and the clip passed through in the next version. So far the strategy is to either just pass through a clip (Cache, InternalCache, and Crop in the next version) or simply throw the IScriptEnvironment::NotFound exception (just like avisynth does when a function doesn't exist), which no plugin is prepared to handle for such a basic function so it escapes out out the plugin.

[wOxxOm]

Do you get any warnings about frames not being prefetched?Yeah, every frame outputs 'Avisynth Compat: requested frame #### not prefetched, using slow method that may deadlock'

And can you share your converted script and the exact plugisn you used an I will take a look at it.I've added it in that message above under 'ported', and oh well here: the script (http://www.privatepaste.com/db2acec810) and plugins (http://puu.sh/13T0Q). Any SD DVD content will do, I suppose.

[Myrsloik]

Yeah, every frame outputs 'Avisynth Compat: requested frame #### not prefetched, using slow method that may deadlock'

I've added it in that message above under 'ported', and oh well here: the script (http://pastebin.com/vuHjKtaK) and plugins (http://puu.sh/13T0Q). Any SD DVD content will do, I suppose.

It's caused by TDecimate. When an avisynth plugin doesn't have a proper list of frames to prefetch it can become very slow. It's a known issue and I'll poke it some more. Did you try yahr by itself?

[wOxxOm]

Did you try yahr by itself?
yes, 32% cpu for mpeg2source+yahr. That is 2% more...
I'd try on a synthetic BlankClip but unlike avisynth's it cannot _create_ new clips, though probably it won't influence the result since freezing one frame doesn't change much except there is no 'frame not prefetched' message: v = yahr (core.std.loop (clip=core.std.trim (clip=v, first=1000, length=1), times=500))

Also lack of chaining makes stuff look obscure (not using 'code' tag as a line this long breaks layout of whole page):

core.avs.mt_makediff (c1=clip, c2=core.avs.removegrain (c1=core.avs.removegrain (c1=core.avs.removegrain (c1=clip, mode=11, modeU=rg11), mode=20, modeU=rg20), mode=20, modeU=rg20), U=uv2, V=uv2)

vs

mt_makediff (clp,clp.removegrain(11,rg11).removegrain(20,rg20).removegrain(20,rg20),U=uv2,V=uv2)

Currently to keep it obvious one needs to serialize calls like this:

v = core.avs.removegrain(c1=clip,mode=11,modeU=rg11)
v = core.avs.removegrain(c1=v,mode=20,modeU=rg20)
v = core.avs.removegrain(c1=v,mode=20,modeU=rg20)
v = core.avs.mt_makediff(c1=clip,c2=v,U=uv2,V=uv2)

[Myrsloik]

yes, 32% cpu for mpeg2source+yahr. That is 2% more...
I'd try on a synthetic BlankClip but unlike avisynth's it cannot _create_ new clips, though probably it won't influence the result since freezing one frame doesn't change much except there is no 'frame not prefetched' message: v = yahr (core.std.loop (clip=core.std.trim (clip=v, first=1000, length=1), times=500))


Uh, exactly how did you measure it? on my computer it at least reaches 90% on all 4 cores when simply having a quick look at the task manager.

[wOxxOm]

Uh, exactly how did you measure it? on my computer it at least reaches 90% on all 4 cores when simply having a quick look at the task manager.Task manager too, i7, 8 cores (4 with HT). Changing threads= parameter in Core() has no effect.

[Myrsloik]

Task manager too, i7, 8 cores (4 with HT). Changing threads= parameter in Core() has no effect.
How do you run the script? I always output to stdout and then do
Python test.py > NUL

[wOxxOm]

yeah, either that or simply v.output(open(os.devnull,'w'))
it works ~10% faster than exactly the same script in avisynth though, where it consumes only 12.5% cpu (1/8 of 100% btw) which is about 2 times less.

[Myrsloik]

Then I have no idea what makes our results so different.
Anyway, expect an update tomorrow with various minor fixes and maybe some new functionality. Dither mvtools and dgmpgdec have been fixed already.

[Keiyakusha]

Anyway, have some performance comparisons from r3 while you wait:
Avisynth 2.5.8
$ time avs2yuv.exe test.avs - > NUL
test.avs: 640x368, 24000/1001 fps, 5000 frames

real 1m23.680s
user 0m0.000s
sys 0m0.015s

VapourSynth r3:
$ time /c/Python32/python.exe test.py > NUL

real 0m23.753s
user 0m0.015s
sys 0m0.000s


Always wanted to ask but keep forgetting every time. This was measured with exactly what mvtools version? and avisynth 2.5.8 was multithreaded or not?

For example for me using single-threaded avisynth with normal mvtools I get speed X. Using Dither-mvtools it can be up to 2X.
But using multithreaded avisynth with 4 threads, SVP-mvtools (just their mvtools, not the actual SVP with GPU vector search) performs even faster than Dither-mvtools at same 4 threads, twice as fast.

[Myrsloik]

Always wanted to ask but keep forgetting every time. This was measured with exactly what mvtools version? and avisynth 2.5.8 was multithreaded or not?

For example for me using single-threaded avisynth with normal mvtools I get speed X. Using Dither-mvtools it can be up to 2X.
But using multithreaded avisynth with 4 threads, SVP-mvtools (just their mvtools, not the actual SVP with GPU vector search) performs even faster than Dither-mvtools at same 4 threads, twice as fast.

It was measured with the single threaded avisynth 2.5.8. The only version I think is good enough for general use. I used the official build of mvtools from fizicks website. Feel free to compare anything you want and post here. I still think it's interesting to see how vapoursynth compares to avisynth Messy Threads.

[Myrsloik]

R6 is here. It fixes several avisynth compatibility issues, most notably dither mvtools and dg decoding stuff.

[sneaker_ger]

Will VapourSynth be video-only?

[Myrsloik]

Will VapourSynth be video-only?

Yes, unless someone is willing to pay me to implement a similar companion module for audio.

[Chikuzen]

on vapoursynth-r6

>>> import vapoursynth as vs
>>> c = vs.Core()
>>> c.avs.LoadPlugin(path='rawsource.dll')
>>> clip = c.avs.RawSource(file='soccer.y4m')
>>> print(clip.width, clip.height)
704 576
>>> cropped = c.std.CropAbs(clip=clip, x = 0, y = 0, width=320. height=240)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "vapoursynth.pyx", line 507, in vapoursynth.Function.__call__ (cython\vapoursynth.c:8864)
vapoursynth.Error: 'CropAbs: negative cropping dimensions not allowed'
>>> cropped = c.std.CropAbs(clip=clip, x = 0, y = 0, width=-320. height=-240)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "vapoursynth.pyx", line 507, in vapoursynth.Function.__call__ (cython\vapoursynth.c:8864)
vapoursynth.Error: 'CropAbs: negative cropping dimensions not allowed'
>>> cropped = core.std.CropAbs(clip=clip, x=0, y=0, width=320, height=-240)
>>> print(cropped.width, cropped.height)
320 -240


:confused::confused::confused::confused::confused::confused::confused::confused::confused::confused::confused:

[Reel.Deel]

I tried nnedi3 and the deinterlacing part works just fine but nnedi3_rpow2 has a problem.

core.avs.nnedi3_rpow2(c1=src,rfactor=2)

Invoke not fully implemented, tried to call: TurnRight but I will pretend it doesn't exist
Avisynth Error: escaped IScriptEnvironment::NotFound exceptions are non-recoverable, crashing...

I'm assuming other parameters like cshift, fwidth, and fheight will be problematic also.

[06_taro]

nnedi3_rpow2 internally invokes AviSynth's resize filter, which cannot be used in VapourSynth.

[Myrsloik]

on vapoursynth-r6
:confused::confused::confused::confused::confused::confused::confused::confused::confused::confused::confused:

I found the typo. Instead of height < 0 it checked height < x.

[Myrsloik]

I tried nnedi3 and the deinterlacing part works just fine but nnedi3_rpow2 has a problem.

core.avs.nnedi3_rpow2(c1=src,rfactor=2)

Invoke not fully implemented, tried to call: TurnRight but I will pretend it doesn't exist
Avisynth Error: escaped IScriptEnvironment::NotFound exceptions are non-recoverable, crashing...

I'm assuming other parameters like cshift, fwidth, and fheight will be problematic also.

I'd rather make a proper port of it than make an effort at fully implementing avisynth's invoke functionality including exact copies of the internal functions it calls.

[cretindesalpes]

Anyway the "_rpow2" part is just an avs script written in C++, wrapping nnedi() and other built-in functions, nothing more. So there would be no gain trying to port the dll part over porting the equivalent call sequence in scripted language. Same for eedi3_rpow2.

[Chikuzen]

It seems that the Y4M file header which VapourSynth outputs is not attached 'C'(color) tag.
Since it is inconvenient to output YUV422/444 stuff, please attach it.

[Myrsloik]

It seems that the Y4M file header which VapourSynth outputs is not attached 'C'(color) tag.
Since it is inconvenient to output YUV422/444 stuff, please attach it.

Added for all 8bit yuv formats. I guess there's no way to specify 10/16 bit ones.

[Myrsloik]

I've released R7. Changelog in the first post. Download from the website. I've also written a post (http://www.vapoursynth.com/2012/09/r7-the-source-is-slowly-coming/) about what will happen in the immediate future. Discuss it here or in the comment field there.

[mastrboy]

I was just about to ask how many more R# releases we would see before the source became public.
About the licensing, "free for non-commercial use" might scare off some potential developers who use avisynth at work.
I would put my vote for a "free for all usage, source code changes to core (vapoursynth) has to be shared, but plugins can be closed source" kind of license.
(Or something that fits the debian guidelines for open source: http://en.wikipedia.org/wiki/Debian_Free_Software_Guidelines )

[sneaker_ger]

I would put my vote for a "free for all usage, source code changes to core (vapoursynth) has to be shared, but plugins can be closed source" kind of license.

Yes, I was about to say the same.

[Keiyakusha]

If some company will decide to use vapoursynth to make money, why Myrsloik can't make money on them?

[Myrsloik]

I was just about to ask how many more R# releases we would see before the source became public.
About the licensing, "free for non-commercial use" might scare off some potential developers who use avisynth at work.
I would put my vote for a "free for all usage, source code changes to core (vapoursynth) has to be shared, but plugins can be closed source" kind of license.
(Or something that fits the debian guidelines for open source: http://en.wikipedia.org/wiki/Debian_Free_Software_Guidelines )

Yes, closed source plugins will of course be allowed. Anything else would just be crazy.
Free is all good and well, but to take this project all the way to where I want it is a bit more complicated.

Let's look at a relevant example: FFMS2
I released it under the MIT license in 2007. It has been about 2 years since I actively did any coding for it myself. Fortunately other people continued to maintain it (see the changelog for a full listing of these nice people), fixing all the api changes in ffmpeg and then the api changes in BOTH ffmpeg and libav. At once. Some bugs got fixed.

But where did the project go? Did it go anywhere? Did any new functionality really get added? Not really as I see it. Most of what end users celebrate are just improvements to libav/ffmpeg.

This brings me to my point. An open source project without a driving force will just sit there. You'll get some bugfixes and not much else. Organic growth and development is good at overcoming small obstacles but in the end directed development is needed to get bigger features. I want this to go further, I want vapoursynth to one day have good GPU support as well. Because without a GPU aware framework avoiding slow up/downloading to the GPU between filters will never be a possibility. In my world a stable CPU only version of vapoursynth is simply the first step. To make my point even more clear, just think about where Avisynth would be without sh0dan or IanB.

So how does this relate to my proposed license? Implementing this will take quite a bit of time. Time is money. Hence my choice to make it free for personal use but ask for a reasonable licensing fee for commercial use.

Note: I know that there is no such legal term as commercial use but there are other ways to state the same idea

[TurboPascal7]

c = vs.Core(accept_lowercase=True)
c.std.loadplugin(path=r'C:\Buf\vapoursynth\ffms2.dll')
ret = c.ffms2.source(source=r"C:\Buf\vapoursynth\sample.mkv")
print(ret.format)
Output with r7:
Traceback (most recent call last):
File "C:/Buf/vapoursynth/test.py", line 7, in <module>
print(ret.format)
File "vapoursynth.pyx", line 202, in vapoursynth.Format.__str__ (cython\vapoursynth.c:3291)
AttributeError: 'str' object has no attribute 'decode'
Works right with r6.

Also, vapoursynth r7 prints the name of every positional argument used to stdout. Looks like someone forgot to remove debug code or something. >__>

[Caroliano]

The most restrictive free software license you can chose is likely is the GNU Affero GPL, and may be a good choice because there is lots of transcoding going on in the ~cloud~. x264 devs apparently managed to get corporate sponsors with GPL, and LuaJIT with the liberal MIT license.

Anyway, the only thing you should not do (besides not releasing the source code ;) ) is create your own open source license. Repositories like Sourceforge and Google Code refuse to host software that are not under a OSI accepted license, exactly to tack on the license proliferation problem.

[Myrsloik]

Also, vapoursynth r7 prints the name of every positional argument used to stdout. Looks like someone forgot to remove debug code or something. >__>

I'v reuploaded R7 with the debug print and .decode stuff fixed.

[Reel.Deel]

Hi Myrsloik, I was wondering if core.list_functions() is the only way to display internal functions.
The reason I ask is because I see people using print(clip.format,clip.width,clip.height) and that does not show up in the core.list_functions().

[JEEB]

Hi Myrsloik, I was wondering if core.list_functions() is the only way to display internal functions.
The reason I ask is because I see people using print(clip.format,clip.width,clip.height) and that does not show up in the core.list_functions().
Print (http://docs.python.org/py3k/library/functions.html?highlight=print#print) is a standard Python function. Now we have the power of a whole scripting language as well at our hands :)

[Reel.Deel]

Ahhh, I see. I guess after work I will be doing some reading. Thanks JEEB. :)

[gyth]

(unfortunately python itself has one huge mutex called the GIL so if it’s used too much it could lead to serious slowdowns)
The GIL is part of the threading module.
There is also a multiprocess (http://docs.python.org/library/multiprocessing.html#module-multiprocessing) module.

I haven't used either so it might be a non solution as well.

[Chikuzen]

If I type 'print(clip.format)' to Python shell, it will be displayed as follows.

Id: 3000024
Name: YUV444P16
Color Family: YUV

But, 'print(help(vs))' says as follows.

Yuv = 3000000
Yuv410P8 = 3000013
Yuv411P8 = 3000014
Yuv420P10 = 3000019
Yuv420P16 = 3000022
Yuv420P8 = 3000010
Yuv420P9 = 3000016
...

This is troublesome.
According to PEP8 (http://www.python.org/dev/peps/pep-0008/#constants), you should unify all the constant names with the capital letter.

[Myrsloik]

If I type 'print(clip.format)' to Python shell, it will be displayed as follows.


But, 'print(help(vs))' says as follows.

This is troublesome.
According to PEP8 (http://www.python.org/dev/peps/pep-0008/#constants), you should unify all the constant names with the capital letter.

Will change it in the next version.

[TheProfileth]

The GIL is part of the threading module.
There is also a multiprocess (http://docs.python.org/library/multiprocessing.html#module-multiprocessing) module.

I haven't used either so it might be a non solution as well.

Hmm after a cursory glance over some of the stuff in there it is pretty interesting. I wonder if Myrsloik had already considered that method?
In an only semi-related question, it has been established that some form of multithreading will be used, but is the multithreading going to be only in the vapoursynth core or independently implemented into individual filters or both or what?
Sorry if this has been covered already just have been under the assumption that vapoursynth itself is multithreaded but individual filters will need to be programmed to support multithreading and I wanted to be clear if this has already been discussed.

[Myrsloik]

Maybe a threading explanation would be in place. First consider avisynth. With its model every filter needs to implement multithreading itself. And to ever get close to 90% cpu usage every filter has to do it really well. Splitting the workload of a single frame evenly for every core.

This has a couple of problems. One is that few people bother to do it. Another is that even fewer have a good understanding of thread synchronization. It can also be fairly error prone and writing portable code that uses threads also requires some extra library for abstraction.

Instead it becomes much simpler when the threading is handled by the core. For example writing a filter that can process several frames at once only needs one extra rule compared to a serial one in avisynth. Don't modify the shared instance data. That's it. Even the slowest filters can usually be modified with not too much effort to match this. The core can now just assign one frame to each thread for processing.

There should be no real reason for filter writers to handle threading themselves. Actually it would most likely trip up the threading handled by the vapoursynth core.

So to sum it up. Vapoursynth filters as such have to know nothing of threading. They just have to follow a few simple rules that makes it possible to run them in a threaded environment.

I hope this makes sense. I'm so sleepy

[TheProfileth]

Thanks for the clarification Myrsloik, that makes a lot more sense

[Rumbah]

Will there be some kind of threading pool for temporal filters?

[Myrsloik]

Will there be some kind of threading pool for temporal filters?

It's already implemented. See how fast mvtools is now. To the core there's no real difference between spatial and temporal filters.

[Chikuzen]

after some consideration, I decided to write a small plugin for VapourSynth.

coding was finished in half a day. LINK (https://github.com/chikuzen/VS_AvsReader)
Since some changes will be further made before a formal release as for VapourSynth and the license of VS is not clear, i will not distribute binary at soon:p

[Myrsloik]

after some consideration, I decided to write a small plugin for VapourSynth.

coding was finished in half a day. LINK (https://github.com/chikuzen/VS_AvsReader)
Since some changes will be further made before a formal release as for VapourSynth and the license of VS is not clear, i will not distribute binary at soon:p

That could definitely be useful. Now all we need is a VSReader for avisynth and then the insane filtering can begin for real :sly:

[tebasuna51]

...
src = ret
c = core.avs.MSuper(c1=src)
b1v = core.avs.MAnalyse(c1=c, delta=1, isb=False)
f1v = core.avs.MAnalyse(c1=c, delta=1, isb=True)
b2v = core.avs.MAnalyse(c1=c, delta=2, isb=False)
f2v = core.avs.MAnalyse(c1=c, delta=2, isb=True)
ret = core.avs.MDegrain2(c1=src, c2=c, c3=b1v, c4=f1v, c5=b2v, c6=f2v)
ret.output(sys.stdout, y4m=True)

Is correct your script?

In MAnalyse docs:
isb : allows to choose between a forward search (motion from the previous frame to current one) for isb=false and a backward search (motion from following frame to the current one) for isb=true (isb stands for "IS Backward", it is implemented and named exactly as written here, do not ask :-). Default isb=false.

The name don't match, but c3 in MDegrain2 must be the backward or the forward?

[Myrsloik]

Is correct your script?

In MAnalyse docs:


The name don't match, but c3 in MDegrain2 must be the backward or the forward?

Congratulations! you're the first once to spot that error!

On a side note, I've decided to use the affero GPL and I'm about halfway to having R8 ready.

[TheProfileth]

Congratulations! you're the first once to spot that error!

On a side note, I've decided to use the affero GPL and I'm about halfway to having R8 ready.

Congrats on your decision :)
Also so I assume with your clarification of the multithreading that you are not considering implementing the multiprocessing module.

[Guest]

On a side note, I've decided to use the affero GPL. Will Affero GPL allow dynamic invocation of Vapoursynth without "infecting" the application, i.e., is it like LGPL? If it is not like LGPL, I submit that it may be an unwise decision to use this license.

Assuming that you will allow some kind of LGPL-like licensing, consider this:

In order not to inhibit plugin writers who wished to keep their plugin source private, and other apps that wanted to use Avisynth without disclosing their source, Ben Rudiak-Gould exempted avisynth.h from LGPL and made it freely usable. Will you consider doing the same for your equivalent interface? Thank you.

[Myrsloik]

Will Affero GPL allow dynamic invocation of Vapoursynth without "infecting" the application, i.e., is it like LGPL? If it is not like LGPL, I submit that it may be an unwise decision to use this license.

Assuming that you will allow some kind of LGPL-like licensing, consider this:

In order not to inhibit plugin writers who wished to keep their plugin source private, and other apps that wanted to use Avisynth without disclosing their source, Ben Rudiak-Gould exempted avisynth.h from LGPL and made it freely usable. Will you consider doing the same for your equivalent interface? Thank you.

Of course I'll have an exception similar to avisynth.

[Daemon404]

Will Affero GPL allow dynamic invocation of Vapoursynth without "infecting" the application, i.e., is it like LGPL? If it is not like LGPL, I submit that it may be an unwise decision to use this license.

Assuming that you will allow some kind of LGPL-like licensing, consider this:

In order not to inhibit plugin writers who wished to keep their plugin source private, and other apps that wanted to use Avisynth without disclosing their source, Ben Rudiak-Gould exempted avisynth.h from LGPL and made it freely usable. Will you consider doing the same for your equivalent interface? Thank you.

Ahem.

http://forum.doom9.org/showpost.php?p=1591132&postcount=150
http://www.vapoursynth.com/2012/09/r7-the-source-is-slowly-coming/

http://i.imgur.com/63mme.png

[Guest]

Well, Daemon404, that's very cute but you missed my point, which I tried to make diplomatically.

Actually, Affero is GPL-like and so is unsuitable for Vapoursynth. I'm interested in the author's opinion on that.

[Daemon404]

Well, Daemon404, that's very cute but you missed my point, which I tried to make diplomatically.

Actually, Affero is GPL-like and so is unsuitable for Vapoursynth. I'm interested in the author's opinion on that.

The two thinks I lined were actually relevant. They make is stance on plugins pretty clear. Also he replied above.

[Guest]

The main point has not been addressed. A GPL-like license does not appear suitable. Please do me a favor and let Myrsloik answer about that if he cares too. Thanks!

[Gavino]

Is correct your script?

The name don't match, but c3 in MDegrain2 must be the backward or the forward?
Actually, although the script is technically incorrect, it doesn't matter here because MDegrain works symmetrically, using both backward and forward vectors in the same way.

[Myrsloik]

The main point has not been addressed. A GPL-like license does not appear suitable. Please do me a favor and let Myrsloik answer about that if he cares too. Thanks!

How would it be unsuitable? Avisynth has been doing fine for a long time with gpl+linking exception. I don't see what new problems having licensing pretty much identical to avisynth would cause. Or is there a huge problem with the avisynth licensing too that I've missed?

[vcmohan]

When you have the Python installed why Visual C++ 2008 and 2010 are required? Are these two only for plugin coders only or ordinary users also are required to install these two?

[active1]

after some consideration, I decided to write a small plugin for VapourSynth.

coding was finished in half a day. LINK (https://github.com/chikuzen/VS_AvsReader)
Since some changes will be further made before a formal release as for VapourSynth and the license of VS is not clear, i will not distribute binary at soon:p

how to use it?

[Myrsloik]

When you have the Python installed why Visual C++ 2008 and 2010 are required? Are these two only for plugin coders only or ordinary users also are required to install these two?

Python needs the vs2008 runtime and vapoursynth needs the 2010 one. I don't see what else there is to say. I do however think python 3.2 installs the 2008 one and I suspect the soon to be released python 3.3 will use the 2010 runtime.

I don't see the problem with this unless there's a misunderstanding somewhere.

[hajj_3]

python 3.3 does indeed use VS2010.

[Chikuzen]

how to use it?

I added readme to git repo previously.
see it.

[Guest]

How would it be unsuitable? Avisynth has been doing fine for a long time with gpl+linking exception. I don't see what new problems having licensing pretty much identical to avisynth would cause. Or is there a huge problem with the avisynth licensing too that I've missed? That would be fine! But originally you only mentioned Afero GPL. Avisynth ships with a GPL license for use of its code and an LGPL license for linking. But I am not aware of any Afero LGPL like license. So I was asking how you intended to manage the linking part. Not trying to make any trouble, just trying to help ensure maximum success!

[malmsteen81]

but it can work for real-time play like ffdhow + avisynth?

[Myrsloik]

but it can work for real-time play like ffdhow + avisynth?

Same method, same hack.

Anyway, the next version is almost ready. I'm surprised none of you pointed out the huge memory leak in crop.

[malmsteen81]

Same method, same hack.



what do you mean?

[active1]

how will the vfwvs work on linux? is it for windows only?

[Myrsloik]

how will the vfwvs work on linux? is it for windows only?

Hint (http://en.wikipedia.org/wiki/Video_for_Windows)

[Chikuzen]

>>> print(clip.format)
Format Descriptor
Id: 1000010
Name: Gray8
Color Family: Gray
Sample Type: Integral
Bits Per Sample: 8
Bytes Per Sample: 1
Planes: 1
Subsampling W: 0
Subsampling H: 0

>>> file=open('gray.yuv', 'wb')
>>> clip.output(file, y4m=True)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "vapoursynth.pyx", line 294, in vapoursynth.VideoNode.output (cython\vapoursynth.c:5193)
vapoursynth.Error: 'Cannot apply y4m headers to non-yuv/unknown formats'

>>> clip.output(file, y4m=False)
>>> file.close()
>>>

https://dl.dropbox.com/u/19797864/y4mheader_exists.png

is this bug already fixed?

[Myrsloik]

>>> print(clip.format)
Format Descriptor
Id: 1000010
Name: Gray8
Color Family: Gray
Sample Type: Integral
Bits Per Sample: 8
Bytes Per Sample: 1
Planes: 1
Subsampling W: 0
Subsampling H: 0

>>> file=open('gray.yuv', 'wb')
>>> clip.output(file, y4m=True)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "vapoursynth.pyx", line 294, in vapoursynth.VideoNode.output (cython\vapoursynth.c:5193)
vapoursynth.Error: 'Cannot apply y4m headers to non-yuv/unknown formats'

>>> clip.output(file, y4m=False)
>>> file.close()
>>>

https://dl.dropbox.com/u/19797864/y4mheader_exists.png

is this bug already fixed?

Will fix. Forgot that gray is also a kind of yuv

[Myrsloik]

R8 released. The post on the front page of the website explains how the new function type can be used.

[kolak]

When vfw module get released how we will be able to use it?

[Myrsloik]

When vfw module get released how we will be able to use it?

By following the instructions.

[kolak]

Hehehe- but what will it give to me?
Will it be a filter which will show up eg, in Vdub?
Will it work as a source filter/decoder with additional processing possibilities?

[Chikuzen]

@Myrsloik
Currently, VS has no methods to convert YUV to RGB.
output/get_frame from COMPATBGR32 cause clash(since it seems that you don't like packed format, I suspect whether this is intentional), and swscale can't convert to GBRP.

I would like to know whether you have a schedule to implement it.

[JEEB]

Hehehe- but what will it give to me?
Will it be a filter which will show up eg, in Vdub?
Will it work as a source filter/decoder with additional processing possibilities?
It's like reading a fake AVI, LikeItSaysOnTheTin.

[Myrsloik]

@Myrsloik
Currently, VS has no methods to convert YUV to RGB.
output/get_frame from COMPATBGR32 cause clash(since it seems that you don't like packed format, I suspect whether this is intentional), and swscale can't convert to GBRP.

I would like to know whether you have a schedule to implement it.

Yes, I intend to keep the core simple. Packed formats are only allowed for special plugins (avisynth compat+the other internal filters where it makes sense). Everything else will be planar. Always. I've asked the libav people about when planar RGB will be implemented. If it's not really soon I'll just write a patch for it myself.

I also found a bug in the resizer and the error handling and fixed it so now at least YUY2<->planar formats should work. And you'll get a nice error message for conversions to RGB24.
Redownload R8 to get the fixes.

[Chikuzen]

Redownload R8 to get the fixes.

now, I can preview a frame via PIL. :thanks:

[kolak]

It's like reading a fake AVI, LikeItSaysOnTheTin.

Well if it's possible to present whole script, with source filter and all processing to external programmes as fake AVI (like current avfs mounting), but with more possible output formats, including v210 etc I will be over the moon- hehe :)

[Are_]

Transpose crashes python.exe
vs-v8(redownload)
python-3.2.3

[Myrsloik]

Transpose crashes python.exe
vs-v8(redownload)
python-3.2.3

That's not much to go on. It crashes with different input formats? Does your cpu have sse2?

[Are_]

Yeah, sorry for my poor report, after further testing:

Input is yv12 coming from native ffms2 (BD m2ts)
As long as the total sum of pixels is less than 2000000, it does not seem to crash (only played a little resizing before Transpose)
Tested on my amd cpu and an i3 friends's one, both with sse2

[Myrsloik]

Yeah, sorry for my poor report, after further testing:

Input is yv12 coming from native ffms2 (BD m2ts)
As long as the total sum of pixels is less than 2000000, it does not seem to crash (only played a little resizing before Transpose)
Tested on my amd cpu and an i3 friends's one, both with sse2

Will be fixed in the next release.

[Myrsloik]

Seems "global" is a special statement to Python, how could I get around this? TIA.


>>> super = core.avs.MSuper(clip, mt=False)
>>> bvec2 = core.avs.MAnalyse(super, isb=True, delta=2, blksize=16, overlap=8, mt=False, truemotion=False, global=True)
File "<stdin>", line 1
bvec2 = core.avs.MAnalyse(super, isb=True, delta=2, blksize=16, overlap=8, mt=False, truemotion=False, global=True)
^
SyntaxError: invalid syntax
>>>


You can put all the arguments in a dict and then pass the dict as function(**dict)
I'll think about what to do with it in the next version. Maybe some kind of automatic argument renaming would be a good idea...

[-Vit-]

Finally had time to try some scripting, ultimate aim to get (Q)TGMC working. Hit some problems:

- No Bob, Merge or TemporalSmooth
-- I cannot replace Bob with the SeparateFields/Resize equivalent because the resizers don't support float sample offsets as they do in Avisynth
-- Merge can be replaced with masktools calls, but will be slower
-- TemporalSmooth can be kinda replaced with trim and masktools work, but will be much slower and really needs scene change awareness. Any suggestions for an existing external plugin that does the same (MVTools can do it with null vectors, but that seems overkill), or does it need a new plugin to be written?

- I have a problem with Interleave (unless I'm being stupid). This script should do nothing, but actually returns a clip with (almost?) every other frame dropped:

#...
e = core.std.SelectEvery(clip=c,cycle=2,offsets=[0]) # SelectEven
o = core.std.SelectEvery(clip=c,cycle=2,offsets=[1]) # SelectOdd
c = core.std.Interleave(clips=[e,o])
#...


- Changelog says unnamed parameters are now accepted, how?

[hajj_3]

python 3.3 will be out within 1 week btw, RC3 is currently out so you can switch to VS2010 once that is released :)

[Myrsloik]

Finally had time to try some scripting, ultimate aim to get (Q)TGMC working. Hit some problems:

- No Bob, Merge or TemporalSmooth
-- I cannot replace Bob with the SeparateFields/Resize equivalent because the resizers don't support float sample offsets as they do in Avisynth
-- Merge can be replaced with masktools calls, but will be slower
-- TemporalSmooth can be kinda replaced with trim and masktools work, but will be much slower and really needs scene change awareness. Any suggestions for an existing external plugin that does the same (MVTools can do it with null vectors, but that seems overkill), or does it need a new plugin to be written?

- I have a problem with Interleave (unless I'm being stupid). This script should do nothing, but actually returns a clip with (almost?) every other frame dropped:


- Changelog says unnamed parameters are now accepted, how?


e = core.std.SelectEvery(c,2,[0]) <- that's all there is to it, simple


It's well known that the resizers suck. I'm looking into several different options to fix it.

You're right. No bob. Maybe you can use some other bob filter and tweak the settings to make it really stupid.

I plan to start work on a kind of masktools replacement based on asmjit after R9 is released, it should be a good base for making the merge functionality you want.

Temporalsmooth is also just another temporal smoother. I don't think it belongs as a base filter.

Maybe someone will come along and rip out some of avisynth's more popular filters and make them standalone plugins. I think it's the best interim solution.

I found the interleave bug. A complete brainfart at its best when it was written.

[-Vit-]

e = core.std.SelectEvery(c,2,[0]) <- that's all there is to it, simple

I tried that and get this:
TypeError: __call__() takes exactly 0 positional arguments (3 given)

[Myrsloik]

You most likely have an old version lying around without realizing it. Delete all copies of vapoursynth.pyd and vscore.dll and reinstall.

[-Vit-]

So I did. I should know better...

[Mug Funky]

here's a replacement for bob, at least using avisynth code (i haven't had time to teach myself python yet):

interleave(separatefields().selecteven().bicubicresize(last.width,last.height,1/3.,1/3.,0,.25,last.width,last.height/2),separatefields().selectodd().bicubicresize(last.width,last.height,1/3.,1/3.,0,-.25,last.width,last.height/2))

i think vapoursynth should have enough internal filters to implement the above line, if the resizers are up to it.

[AzraelNewtype]

here's a replacement for bob, at least using avisynth code (i haven't had time to teach myself python yet):

interleave(separatefields().selecteven().bicubicresize(last.width,last.height,1/3.,1/3.,0,.25,last.width,last.height/2),separatefields().selectodd().bicubicresize(last.width,last.height,1/3.,1/3.,0,-.25,last.width,last.height/2))

i think vapoursynth should have enough internal filters to implement the above line, if the resizers are up to it.

The resizers only handle integer offset values, it's literally the problem -Vit- said he had with trying to implement it by hand himself.

[Myrsloik]

I've released R9. See http://www.vapoursynth.com/2012/09/r9-the-mostly-done-release/ for the full post with changes and what's on my todo list.

Here's a small example of the new slicing possible with clips in Python:
clip[1:] # equivalent to Trim(c, first=1)
clip[:9] # equivalent to Trim(c, last=8)
clip[1::2] #equivalent to selecting all frames with odd numbers
clip[::-1] # Reverse(c)
Interleave([clip[::2], clip[1::2]]) # do "nothing"
clip1[200::-5] + clip2[5:10:3] # ponder this one

[kolak]

Can't wait to see vfw module, even if I'm still not sure how useful it will be :)

[Myrsloik]

The slicing result seems to be wrong or do i do anything wrong :confused:

You found the single case where I forgot to adjust the trim end appropriately (with only stop and not start set). I've reuploaded R9 with that small detail fixed.

Note that clip[0:9] will give you the correct result even in the broken version.

[-Vit-]

Trying to do some temporal scripting. Couple of queries:
- SelectEvery fails if any selected frame is negative (doesn't seem to be so much a problem at the clip end...?). Avisynth duplicates the start/end frames, so allowing "out-of-cycle" offsets. This is used by many avisynth scripts. Without it, temporal processing at the clip ends becomes trickier.
- Related, SelectEvery does not accept a cycle of 1. For selecting the clip offset by a number of frames: SelectEvery(c,1,[-2])

- Negative slice start and end values? I thought they counted from the end of the sequence. Seems to reject them here. Exacerbates the above two points because the workarounds involve padding at the start and the end.

- Suggestion: * operator for clip repetition. c * 2 = clip repeated twice.

[Myrsloik]

Trying to do some temporal scripting. Couple of queries:
- SelectEvery fails if any selected frame is negative (doesn't seem to be so much a problem at the clip end...?). Avisynth duplicates the start/end frames, so allowing "out-of-cycle" offsets. This is used by many avisynth scripts. Without it, temporal processing at the clip ends becomes trickier.
- Related, SelectEvery does not accept a cycle of 1. For selecting the clip offset by a number of frames: SelectEvery(c,1,[-2])

- Negative slice start and end values? I thought they counted from the end of the sequence. Seems to reject them here. Exacerbates the above two points because the workarounds involve padding at the start and the end.

- Suggestion: * operator for clip repetition. c * 2 = clip repeated twice.

Trim splice and repeat is for offsetting a clip. Why you need selectevery to do so is a bit of a mystery to me. Seems like you're stuck in your avisynth ways. I suppose I could allow out of cycle offsets in selectevery if you have a good use case.
Also, negative offsets in slicing is just crazy. It goes against all that is good and python. Use clip[clip.num_frames - 10, clip.num_frames - 5, -1] If you need something similar. I intentionally kept trim simple. Look at the mess in avisynth and you'll see why.
I'll think about a repeat operator but it's a lot less common for someone to need it. I want to keep it simple and avoid operator overloading hell.

[-Vit-]

I'm was trying to do a 2-radius temporal smooth in script for TGMC. The trick is the clip ends, where there aren't enough frames. Naturally I started with the avisynth approach because it deals with that trivially; and there are no complex vapoursynth script examples - you gotta start somewhere.

Also, negative offsets in slicing is just crazy. It goes against all that is good and python.
I rarely code Python so I don't have that Pythony mindset... but negative indices are allowed in ordinary Python list slicing so why are they not good Python?

I'll think about a repeat operator but it's a lot less common for someone to need it. I want to keep it simple and avoid operator overloading hell.
It's useful for the padding needed at the ends of the clip I needed to implement the above. I guess it can be done with slice+loop. But you go against expectations by providing some list operators and not others (I immediately went for the * operator and I'm not a Pythoner)

[Myrsloik]

I'm was trying to do a 2-radius temporal smooth in script for TGMC. The trick is the clip ends, where there aren't enough frames. Naturally I started with the avisynth approach because it deals with that trivially; and there are no complex vapoursynth script examples - you gotta start somewhere.


I rarely code Python so I don't have that Pythony mindset... but negative indices are allowed in ordinary Python list slicing so why are they not good Python?


It's useful for the padding needed at the ends of the clip I needed to implement the above. I guess it can be done with slice+loop. But you go against expectations by providing some list operators and not others (I immediately went for the * operator and I'm not a Pythoner)

Right, I missed the part with negative indexes. It will be added. I try to be pythonish whenever possible. I suppose I'll add * too just to see how it turns out.

As for scripts I suggest you post a few lines of avisynth script here and I'll tell you how it'd write it. I'm actually surprised so few people ask me about the syntax.

[lansing]

ok i'lve read all 12 pages of this thread, and i have the script ready in a py file. But how do you run it with different codec like xvid, divx and x264? I mean how do I write it in the command window? And how do I measure the speed of each filters without outputing the video file?

And there's no link in the documentation page that redirect back to the homepage

[Myrsloik]

ok i'lve read all 12 pages of this thread, and i have the script ready in a py file. But how do you run it with different codec like xvid, divx and x264? I mean how do I write it in the command window? And how do I measure the speed of each filters without outputing the video file?

And there's no link in the documentation page that redirect back to the homepage

1. See the first post on how to pipe to x264.exe.
2. Pipe to NUL (script.py > NUL) to not write a file.
3. I usually use time in mingw when just doing quick timing.

[Reel.Deel]

Thanks for the new VapourSynth documentation, it's compact yet very informative.

So I noticed that when using AVS plugins in VS, boolean parameters are now integer. Was this how you intended it to be or is it because of the Python interpreter?

[Myrsloik]

Thanks for the new VapourSynth documentation, it's compact yet very informative.

So I noticed that when using AVS plugins in VS, boolean parameters are now integer. Was this how you intended it to be or is it because of the Python interpreter?

It's a design decision in vs to keep the number of types down so bool was left out and frames and functions added instead.
I've also marked all arguments that work like a bool with bint in the documentation if anyone's wondering what it means.
It has absolutely nothing to do with python.

[Chikuzen]

@Myrsloik
Although it seems that you stop using B tag with Y4M output and added like 420p16 to C tag, I think that this is not good.
C tag also has the role which specifies chroma position other than a colorspace specification.
You should not give the function more than now to C tag.

If you want to process over 8bit format with Y4m, you should use new B tag.
And it will be better to decide that W becomes half if value of B is 9 or more (like dither-hack's interleaved format).
At least, compatibility is not be broken if it carries out like this.
A Y4M purser which does not refer to B tag will only process it as a collapsed image with twice width.

[Myrsloik]

@Myrsloik
Although it seems that you stop using B tag with Y4M output and added like 420p16 to C tag, I think that this is not good.
C tag also has the role which specifies chroma position other than a colorspace specification.
You should not give the function more than now to C tag.

If you want to process over 8bit format with Y4m, you should use new B tag.
And it will be better to decide that W becomes half if value of B is 9 or more (like dither-hack's interleaved format).
At least, compatibility is not be broken if it carries out like this.
A Y4M purser which does not refer to B tag will only process it as a collapsed image with twice width.


Actually that change is to be more compatible. It's how ffmpeg already implemented it so you can pipe directly to it now.

[lansing]

i'm having problem working with ts files using ffms2. It's getting really slow when parsing in a large ts file, like 30G, where indexing took like 15 minutes and encoding speed went down to 2fps. I tested to trim a small portion (4000 frame) of the same ts file, indexing was like instant and speed constant at 40fps. Don't know what is causing the let down.

And I'm getting rare framerate on the output video, like 4382/143 fps, where it supposed to be 25fps.

[Chikuzen]

Actually that change is to be more compatible. It's how ffmpeg already implemented it so you can pipe directly to it now.

oh, I hope libav does not take this in.

[Myrsloik]

i'm having problem working with ts files using ffms2. It's getting really slow when parsing in a large ts file, like 30G, where indexing took like 15 minutes and encoding speed went down to 2fps. I tested to trim a small portion (4000 frame) of the same ts file, indexing was like instant and speed constant at 40fps. Don't know what is causing the let down.

And I'm getting rare framerate on the output video, like 4382/143 fps, where it supposed to be 25fps.

This is a known ffms2 issue. It has never been good with transport streams.

[Myrsloik]

oh, I hope libav does not take this in.

I intend to port it soon. Actually it's already a mess because y4m only has different color positions defined for yuv420p8. Anything else and it doesn't. I'd say this is the proper way would be to add flags for colorimetry, range and chroma position in new fields. But y4m is already a mess and nobody would support it properly anyway...

[gnaggnoyil]

It seems that TFM and TDecimate can't work normally and properly under vapoursynth. Is there any replaceable filters to do the ITVC?
Since the lgd files I use in delogo filters must work after the ITVC is complete, it would be very helpful for me to solve the problem.
Or perhaps I had to do it in mencoder? lol.

--------------------

I use TDM and TDecimate like:

video=core.avs.TFM(c1=video,order=1)
video=core.avs.TDecimate(c1=video,mode=1,hybrid=1)

and in the command prompt the script warns like this:

Avisynth Compat: requested frame XXX not prefetched, using slow method that may deadlock

[Myrsloik]

I use TDM and TDecimate like:

video=core.avs.TFM(c1=video,order=1)
video=core.avs.TDecimate(c1=video,mode=1,hybrid=1)

and in the command prompt the script warns like this:

Avisynth Compat: requested frame XXX not prefetched, using slow method that may deadlock

It's a warning, not an error. Try reading next time.

[gnaggnoyil]

It's a warning, not an error. Try reading next time.

:scared:But it seems there is something wrong which is slowing the speed of the script, any method to solve it?

[Myrsloik]

:scared:But it seems there is something wrong which is slowing the speed of the script, any method to solve it?

Start porting TDecimate. That's the solution. The filter has really odd frame request patterns and because of that it's impossible to prefetch all the input frames it needs.
You can also try your luck with TFM+Decimate from decomb. It may actually end up a bit faster in vs.
But yeah... tritical makes filters with good ideas but very messy code. Porting these things will take time.

[Chikuzen]

:scared:But it seems there is something wrong which is slowing the speed of the script, any method to solve it?

#script_1.py
import vapoursynth as vs
import sys
core = vs.Core(threads=4)
core.avs.LoadPlugin("TIVTC.dll")
core.avs.LoadPlugin("DGDecode.dll")
video = core.avs.MPEG2Source("foo.d2v", idct=3)
video = core.avs.TFM(video, order=1)
video = core.avs.TDecimate(video, mode=1,hybrid=1)
video.output(sys.stdout, y4m=False)


$time python3 script_1.py > /dev/null 2>&1
real 2m1.805s
user 0m0.000s
sys 0m0.015s


#script_2.py
import vapoursynth as vs
import sys
core = vs.Core(threads=4)
core.std.LoadPlugin("vsavsreader.dll")
lines = '''
LoadPlugin("TIVTC.dll")
LoadPlugin("DGDecode.dll")
MPEG2Source("foo.d2v", idct=3)
TFM(order=1)
TDecimate(mode=1,hybrid=1)
'''
video = core.avsr.Eval(lines)
video.output(sys.stdout, y4m=False)


$ time python3 script_2.py > /dev/null
real 1m44.076s
user 0m0.000s
sys 0m0.045s


you can get speed comparable as avisynth when using vsavsreader.

[Myrsloik]

I've released R10. Enjoy the vfw stuff. Just remember that vs does not do error message clips, instead opening the file fails. Only applications like vdub that query the error information interface and the python command line will show the actual error message.

Also, please consider donating to the effort of making a complete cross platform frameserver.

Some more information and details in my blog post here:
R10 – VFW enters the scene from the left (http://www.vapoursynth.com/2012/09/r10-vfw-enters-the-scene-from-the-left/)

[lansing]

this is just amazing to see how fast vapoursynth grow in just a month, I foresaw the dimming of avisynth within the next two months at this pace. Hats to you Myrsloik

[lansing]

I just updated to r10, but can't seems to get the preview working.

I renamed the py file to vpy, followed the instruction to open in virtualdub, but got a out of bound memory crash after opening it. And MPC-HC status just showing "opening..."
Am I doing something wrong?

[Myrsloik]

I just updated to r10, but can't seems to get the preview working.

I renamed the py file to vpy, followed the instruction to open in virtualdub, but got a out of bound memory crash after opening it. And MPC-HC status just showing "opening..."
Am I doing something wrong?

I forgot to mention the most important part :D
Scripts you want open in this way need to have the returned clip assigned to the variable last and cannot use clip.output().
An example:
ret = source filter here
ret = other filters(ret)
last = ret
Then try to open it in vdub.

[kolak]

Vfw came so fast, but what about software, which does open files by extention? Even if it works with vfw it will fail as extension won't be avi. Making script seen by other apps as "fake" avi (or mov) would be great. This would make it 99% compatible with all apps.
Are 8bit+ color spaces/outputs coming? Again v210 would be great as this is most compatible 10bit format.

I should make a use of this donatation button :)

[lansing]

stilll no luck, still crashed :(


import vapoursynth as vs
import sys
core = vs.Core(accept_lowercase=True)

# plugins
core.avs.LoadPlugin(r"C:\Program Files (x86)\AviSynth 2.5\plugins\DGDecode.dll")

# load source
src = core.avs.MPEG2Source(d2v=r"test.d2v")

last=src

[Myrsloik]

Vfw came so fast, but what about software, which does open files by extention? Even if it works with vfw it will fail as extension won't be avi. Making script seen by other apps as "fake" avi (or mov) would be great. This would make it 99% compatible with all apps.
Are 8bit+ color spaces/outputs coming? Again v210 would be great as this is most compatible 10bit format.

I should make a use of this donatation button :)

For the fake avi stuff you need another layer like that old vfapi stuff or avfs. I could try to quickly port avfs and make it vsfs if the current solution isn't good enough.

I looked at v210 but it's packing made me nauseous just to look at. I suppose I'll add it sooner or later but it's a shame it turned out to be the most popular. Microsoft defines very sane and clean planar formats officially too. What a shame... over 8 bit output will come. But it's a bit lower in priority since to my knowledge x264 is the only thing that truly benefits from it. Correct me if I'm wrong.

[Myrsloik]

stilll no luck, still crashed :(


import vapoursynth as vs
import sys
core = vs.Core(accept_lowercase=True)

# plugins
core.avs.LoadPlugin(r"C:\Program Files (x86)\AviSynth 2.5\plugins\DGDecode.dll")

# load source
src = core.avs.MPEG2Source(d2v=r"test.d2v")

last=src



Then I'm not sure why it fails. Try running the program from the command-line. Maybe you'll see a warning or error printed before it crashes. You can also try blankclip and see if that works.

[Reel.Deel]

VS VFW works fine for me. Thanks Myrsloik.

I'm assuming it will be up to VDub to support drag and drop for vpy or am I wrong?

[Myrsloik]

VS VFW works fine for me. Thanks Myrsloik.

I'm assuming it will be up to VDub to support drag and drop for vpy or am I wrong?

Correct. Vdub needs to be updated to know about the .vpy extension. Feel free to go and push for its inclusion in the next vdub version. Along with support for the sane high bitdepth planar formats.

[Reel.Deel]

Feel free to go and push for its inclusion in the next vdub version. Along with support for the sane high bitdepth planar formats.

Will do. :D

[lansing]

Then I'm not sure why it fails. Try running the program from the command-line. Maybe you'll see a warning or error printed before it crashes. You can also try blankclip and see if that works.

can you elaborate how to run it the command line way?

[Myrsloik]

can you elaborate how to run it the command line way?

Start menu\open command prompt
Cd virtualdub path
Virtualdub

Done.

Another thing you should try first though is only having absolute paths in your script. The other thing is running your script in python and see if it works at all with the paths you have.

[Keiyakusha]

Works fine so far thank you so much! Now I can do some real stuff with it!
But I have somewhat unrelated question.
Should I load ffms2 as std or avs plugin? Using latest ffms2 build it is loaded via std yet says it "doesn't have function FFVideoSource". I think I missed something. How should I call it then?

[Chikuzen]

Works fine so far thank you so much! Now I can do some real stuff with it!
But I have somewhat unrelated question.
Should I load ffms2 as std or avs plugin? Using latest ffms2 build it is loaded via std yet says it "doesn't have function FFVideoSource". I think I missed something. How should I call it them?

http://forum.doom9.org/showpost.php?p=1590957&postcount=1650

[active1]

Correct. Vdub needs to be updated to know about the .vpy extension. Feel free to go and push for its inclusion in the next vdub version. Along with support for the sane high bitdepth planar formats.

maybe someone should add the .vpy extension support in x264 too (is it possible to do it?)
EDIT: sorry, it works :p
EDIT: lol, no it's not working, sorry :stupid:
that's what happend:
avs [info]: trying FFmpegSource2... indexing... failed
avs [info]: trying DSS2... not found
avs [info]: trying DirectShowSource... succeeded
avs [info]: 848x480p 0:0 @ 10000000/417013 fps (cfr)

the x264 loaded the test.vpy script through directshowsource

[Keiyakusha]

http://forum.doom9.org/showpost.php?p=1590957&postcount=1650

Yeah, definitely missed that. Thanks! It works now.

EDIT: one more question, it is not possible to open 10bit video now? It says something about vfw but I can't read it,anything that reads .vpy crashes too fast.
edit2: by the way 10bit useful not only for x264. me, and I believe not only me use it for more professional stuff. some things like 10bit dnxhd are pretty much standard for delivering intermediate files. thankfully after all we got nicely hacked ffms2 for avisynth with high bitdepth and different subsampling support ^__^

[kolak]

For the fake avi stuff you need another layer like that old vfapi stuff or avfs. I could try to quickly port avfs and make it vsfs if the current solution isn't good enough.

I looked at v210 but it's packing made me nauseous just to look at. I suppose I'll add it sooner or later but it's a shame it turned out to be the most popular. Microsoft defines very sane and clean planar formats officially too. What a shame... over 8 bit output will come. But it's a bit lower in priority since to my knowledge x264 is the only thing that truly benefits from it. Correct me if I'm wrong.

vsfs would be great- avfs works very well as it presents script as "normal uncompressed avi", so every app is happy with it. It also works stable.

v210 is definitely the most universal format for 10bit and I think they made it packed to save bits.
Is it possible to have some 10bit working color space in one format (easy to implement and to work with) and have v210 implemented in vsfs only as final output? If this is easier than it would also work.

v210 spec can be found here:

https://developer.apple.com/quicktime/icefloe/dispatch019.html#v210

[JEEB]

v210 is definitely the most universal format for 10bit and I think they made it packed to save bits.
It is the most "universal" format for 10bit in MOV (and thus possibly with the pro tools). Please leave that stuff out of Vapoursynth and pipe its 10bit output to ffmpeg/avconv or something, they can convert the bits into the correct thing, and they can mux it into mov correctly. Myrsloik knows of the format and it's not something he most probably will support any time soon, because it is a major PITA to do so.

Also the v210 "support" in VirtualDub is as far as I know something that doesn't work anywhere else, so that can, if you pardon my french, go suck itself. v210 is only "standard" in mov. I would much rather see the support for the saner, proper 10bit formats be added in VDub instead of a random format not meant for in-program use, but for creating an actual mov file for transport for loading it into something else. And for that, we have ffmpeg/avconv to do it for you. You just have to feed them the 10bit YCbCr.

Thank you.

[active1]

will vapoursynth support outputing more than 8 bits? and will vdub be able to preview the +8bit.vpy script?

[JEEB]

will vapoursynth support outputing more than 8 bits?
Vapoursynth has been able to output >8bit stuff for quite a while already. That is why the Vapoursynth ffms2 plugin can output >8bit stuff as well, which was never officially implemented on the Avisynth side as it was a hack.

and will vdub be able to preview the +8bit.vpy script?
No idea what colorspaces can be used in the VFW interface. Also I have no idea what colorspaces VDub can take in. IIRC it doesn't support the MS-specified 10bit+ YCbCr formats at least. It is an open source application though, so things can be implemented.

[Keiyakusha]

EDIT: one more question, it is not possible to open 10bit video now? It says something about vfw but I can't read it,anything that reads .vpy crashes too fast.

Just want to rephrase this a bit
Lets say I want to open any 10 (or more) bit file supported by ffmpeg using ffms2 in vapoursynth and then preview it in something. How should I do this correctly?
From what I understand VDub needs to add support for planar colorspaces, then how do I convert colorspace to somthing VDub already understands? 8bit YV12 for example

edit: this? Bicubic(clip=clip, format=vs.YUV420P8) why bicubic and not something else? how it is related to colorspace conversion if no subsampling involved? And it seems it crashes anyway.

[active1]

Vapoursynth has been able to output >8bit stuff for quite a while already. That is why the Vapoursynth ffms2 plugin can output >8bit stuff as well, which was never officially implemented on the Avisynth side as it was a hack.

i meant outputing more than 8bit with using the vfw module

.
No idea what colorspaces can be used in the VFW interface. Also I have no idea what colorspaces VDub can take in. IIRC it doesn't support the MS-specified 10bit+ YCbCr formats at least. It is an open source application though, so things can be implemented.
i hope that vapoursynth will be able to do it :)

[Reel.Deel]

then how do I convert colorspace to somthing VDub already understands? 8bit YV12 for example.

This allows you to preview 10-bit video as 8-bit in VDub.
.......
# 10-bit source
src = core.ffms2.source(source=r"C:\10 bit.mkv")

# Change bit depth to 8-bit
src = core.resize.Bicubic(clip=src, format=vs.YUV420P8)

# Ouput for VDub
last = src

Also, you can change Bicubic to any of the other resize listed here (http://www.vapoursynth.com/doc/functions/resize.html).


------------

@ Myrsloik

Is there any dithering going on when converting YUVxxxP16 to lower bit depths?

[hajj_3]

python 3.3 final is out now: http://python.org/download/

Maybe you could release a new build using Visual Studio.NET 2010 now :) I wonder if windows 8 has the C++ 2010 pre-installed?

[JEEB]

@ Myrsloik
Is there any dithering going on when converting YUVxxxP16 to lower bit depths?
Yes, depends on the swscale linked into vapoursynth, after the dither package gets implemented we should get various things for that.

Of course for preview you could just convert it to RGB straight away (and after the dither package gets ported you'd have more ways to do that, too) :)

[Keiyakusha]

This allows you to preview 10-bit video as 8-bit in VDub.

Also, you can change Bicubic to any of the other resize listed

Thanks. I used it a bit wrong. Now it works. I know that I can change resize, but what is the difference? I'm not doing any scaling or chroma conversion.

Is there any dithering going on when converting YUVxxxP16 to lower bit depths?
want to add to JEEB's answer that swscale does poor ordered dithering... at least ffms2 supports only that. it was my request some time ago to add something better

[Reel.Deel]

I know that I can change resize, but what is the difference? I'm not doing any scaling or chroma conversion.

Sorry, must of misunderstood your last post.


Bicubic(clip=clip, format=vs.YUV420P8) why bicubic and not something else?

[Myrsloik]

Thanks. I used it a bit wrong. Now it works. I know that I can change resize, but what is the difference? I'm not doing any scaling or chroma conversion.


want to add to JEEB's answer that swscale does poor ordered dithering... at least ffms2 supports only that. it was my request some time ago to add something better

There most likely is no difference since the scalers probably don't get involved at all.

[lansing]

Another thing you should try first though is only having absolute paths in your script. The other thing is running your script in python and see if it works at all with the paths you have.

Yes, I figured it's the path that's causing it.

how do I write it for the import path if my vpy file is on the desktop, and python was installed at "C:\Python32" ?

______________________________

And a few more things I spotted:

- the uninstaller shortcut was missing in the start menu.
- vapoursynth installer defaulted the python directory as python32. I updated python to 3.3 today and vapoursynth installer throws out an error
- a context menu on creating a new blank vpy file should be added
- there's a pop up sometime asking system reboot after i uninstalling vapoursynth


And I did some benchmarking with x264 preset ultrafast comparing avisynth and vapoursynth.


avs FFVideoSource("test.avi") 524fps x264 cpu usage: 100%
vpy Source("test.avi") 285fps x264 cpu usage: ~50% python.exe cpu usage: 30%


the python.exe cpu usage seems to bottleneck the speed.

[mastrboy]

avs avisource("test.avi") 524fps x264 cpu usage: 100%
vpy ffms2("test.avi") 285fps x264 cpu usage: ~50% python.exe cpu usage: 30%



For the test to be comparable should you not have used FFVideoSource in avisynth rather than avisource?

[Myrsloik]

Yes, I figured it's the path that's causing it.

how do I write it for the import path if my vpy file is on the desktop, and python was installed at "C:\Python32" ?

______________________________

And a few more things I spotted:

- the uninstaller shortcut was missing in the start menu.
- vapoursynth installer defaulted the python directory as python32. I updated python to 3.3 today and vapoursynth installer throws out an error
- a context menu on creating a new blank vpy file should be added
- there's a pop up sometime asking system reboot after i uninstalling vapoursynth


And I did some benchmarking with x264 preset ultrafast comparing avisynth and vapoursynth.


avs avisource("test.avi") 524fps x264 cpu usage: 100%
vpy ffms2("test.avi") 285fps x264 cpu usage: ~50% python.exe cpu usage: 30%


the python.exe cpu usage seems to bottleneck the speed.

1. The shortcut isn't missing. I believe it's actually the guideline of the week to put that stuff in remove programs only.
2. The python module is built for 3.2 and will only work with that version. It's how annoying python on windows is.
3. I'll put it on the to-do list. Personally I never use those options so I don't notice it missing.
4. Of course. A file was in use and a reboot is needed to delete it. It's safe to just install over for upgrades though.

Can you do the same benchmark but pipe the input to x264 and skip the vfw stuff? It'll help me find the bottleneck

[lansing]

For the test to be comparable should you not have used FFVideoSource in avisynth rather than avisource?

typo by me, actually they were both ran with ffvideosource

[lansing]

Can you do the same benchmark but pipe the input to x264 and skip the vfw stuff? It'll help me find the bottleneck

they were done without the vfw thing already, tested well over 50k frames

vpy:

import vapoursynth as vs
import sys

core = vs.Core(accept_lowercase=True)
core.std.LoadPlugin(r'C:\Program Files (x86)\AviSynth 2.5\plugins\ffms2.dll')
src = core.ffms2.Source(r"test.avi")

src.output(sys.stdout, y4m=True)

___________________________________

cmd:
python test.vpy | x264 --preset ultrafast --demuxer y4m --output vpy_ben.mkv -




avs:

ffvideosource("test.avi")

___________________________________

cmd:
x264 --preset ultrafast --output avs_ben.mkv test.avs

[kolak]

It is the most "universal" format for 10bit in MOV (and thus possibly with the pro tools). Please leave that stuff out of Vapoursynth and pipe its 10bit output to ffmpeg/avconv or something, they can convert the bits into the correct thing, and they can mux it into mov correctly. Myrsloik knows of the format and it's not something he most probably will support any time soon, because it is a major PITA to do so.

Also the v210 "support" in VirtualDub is as far as I know something that doesn't work anywhere else, so that can, if you pardon my french, go suck itself. v210 is only "standard" in mov. I would much rather see the support for the saner, proper 10bit formats be added in VDub instead of a random format not meant for in-program use, but for creating an actual mov file for transport for loading it into something else. And for that, we have ffmpeg/avconv to do it for you. You just have to feed them the 10bit YCbCr.

Thank you.

Hmmm?

v210 is supported by almost every single software- from Vdub, through ffmpeg, Premiere, .... ending at high end grading tools and all preview cards- BM, AJA etc.
It works for AVI the same as for MOV- one can be re-wrapped with no problem. Whole broadcast industry uses v210, so it's not something rare. As I said- it's the most supported 10bit format out there (except DPX, but this is an image sequence).
Vdub is happy with v210- don't know about any problems with Vdub and v210 and I use it quite a lot.

I know that it's not a easy format to work with (for processing), that's why I said it can exist only on final output stage. This, what I'm talking about is eg. fake AVI (like current avfs showing up as YUY2), which would have v210 format. This would make it working with 99% existing software- there is no other 10bit format with such a broad support.
It's all about linking script to outside apps at higher than 8bit precision. Current avfs works very well, but only for 8bit. It's also about not have to render big v210 file, but use fake AVI as this is very useful solution. AVFS works very well and I hope there will be 10bit version of it.

[lansing]

ok i was able to solve my problem with the vd crashing from a kind forum member, it was caused by me setting the encoding of my vpy file to UTF-8, changing it back to ANSI solves it.

And since now I got the fake avi working, I will do some benchmarking with AVSMeter on pure avs and vpy in a avs wrapper, will report.

[Pat357]

I still have crashes in Vdub upon opening this script :

import vapoursynth as vs
import sys
core = vs.Core(accept_lowercase=True)
core.std.LoadPlugin(r'k:\programs\ffms2-r725-icl\ffms2.dll')
## src = core.ffms2.Source(r'k:\film\Battleship (2012) HDRip\Battleship.2012.mkv')
ret = core.ffms2.Source(r"k:\film\Battleship (2012) HDRip\Battleship.2012.mkv")
## src.output(sys.stdout, y4m=True)
last = ret

It works if I use "sys.stdout" and pipe the result to x264.

What do I need to modify to open it in vdub/MPC-HC/AVISource ?
Is there a FourCC specified for the VFW format ? Is this needed ?

I've r10 installed (=checked using "print(c.version())")

[lansing]

phaeron from virtualdub confirm (http://forums.virtualdub.org/index.php?act=ST&f=2&t=21254&) adding supports for vpy extension.


And i'm having small issue with argument having float value, like fft3dfilter(sigma=1.0) . I have to add a decimal in order for it to work, if i put "1" it will return invallid argument.

[Keiyakusha]

ok i was able to solve my problem with the vd crashing from a kind forum member, it was caused by me setting the encoding of my vpy file to UTF-8, changing it back to ANSI solves it.

And since now I got the fake avi working, I will do some benchmarking with AVSMeter on pure avs and vpy in a avs wrapper, will report.

By the way, this is interesting question. Can we have unicode support for .vpy files?

Edit: What are the names of planar colorspaces for 4:4:4 8+bit material? I know only these: P016,P010,P216,P210, but this is only 420 and 422. Microsoft website mentions something like Y410, but they are packed? So Vapoursynth have no choice but support something more than planar colorspaces?

[Chikuzen]

EDIT: my miss understanding, deleted

What are the names of planar colorspaces for 4:4:4 8+bit material? I know only these: P016,P010,P216,P210, but this is only 420 and 422. Microsoft website mentions something like Y410, but they are packed? So Vapoursynth have no choice but support something more than planar colorspaces?

It is not a name but FourCC(Identifier) which you have said.
They will be defined if Microsoft, other companies, or an person is needed.
Since Microsoft had not used the YUV444-Planar format yet in MediaFoundation, it is not exists now.

[Keiyakusha]

Edit: i wanted to ask if it possible to open image sequences in VS, but from what I see in documentation just now:ImageReader/ImageWriter none -- Planned, contributions welcomeit seems not. For example I'd like to open sequence of 5000 16bit per component PNG's or PSD's as 24000/1001fps video. I guess we need VS plugin based on DevIL for that. Well, 32bit per component hdr, psd, tiff should be an option too. From what I understand DevIL is capable of doing this.


They will be defined if Microsoft, other companies, or an person is needed.
Since Microsoft had not used the YUV444-Planar format yet in MediaFoundation, it is not exists now.
I see. But is it something that can be defined in vapoursynth? Won't it be better to just implement packed variants that already exist?

[Chikuzen]

Edit: i wanted to ask if it possible to open image sequences in VS
I wrote vsavsreader as a substitute until plugins for VS are developed.
You can use avisynth's ImageSource vis that.

I see. But is it something that can be defined in vapoursynth? Won't it be better to just implement packed variants that already exist?
VS has already defined those format names.
try as follows
>>> import vapoursynth
>>> print(help(vapoursynth))
you will find them at the end of descriptions.

[Keiyakusha]

I wrote vsavsreader as a substitute until plugins for VS are developed.
You can use avisynth's ImageSource vis that.
Yes I know about your handy plugin, thanks. But unfortunately avisynth is not capable of opening high bitdepth image sequences in any way. There is no even hacks for that (or I just never heard of them).
Of course 8bit images is possible, but there is no reason for me to have 8bit sequences, there is nice lossless video formats. There is no good lossless formats for 8+ bitdepth though... At least not the ones that are free and supported by different software

[Myrsloik]

they were done without the vfw thing already, tested well over 50k frames

___________________________________

cmd:
x264 --preset ultrafast --output avs_ben.mkv test.avs
[/code]

I've reproduced it. But I'm not quite sure why it happens. I also redid the comparison with avs2yuv (to add the piping overhead in both cases) and avisynth is then still 50% faster.

[Pat357]

Nice, I have VFW working with MPC & AVISource, but I get access violations with VirtualDUB v1.9.11.
I've tried the newer beta vdub v1.10.2, but got same problem.

Any idea what could be wrong ?
Thanks in advance !

crash report :


VirtualDub crash report -- build 32842 (release)
--------------------------------------

Disassembly:
00000000: 0000 add [eax], al <-- FAULT
00000002: 0000 add [eax], al
00000004: 0000 add [eax], al
00000006: 0000 add [eax], al
00000008: 0000 add [eax], al
0000000a: 0000 add [eax], al
0000000c: 0000 add [eax], al
0000000e: 0000 add [eax], al
00000010: 0000 add [eax], al
00000012: 0000 add [eax], al
00000014: 0000 add [eax], al
00000016: 0000 add [eax], al
00000018: 0000 add [eax], al
0000001a: 0000 add [eax], al
0000001c: 0000 add [eax], al
0000001e: 0000 add [eax], al
00000020: 0000 add [eax], al
00000022: 0000 add [eax], al
00000024: 0000 add [eax], al
00000026: 0000 add [eax], al
00000028: 0000 add [eax], al
0000002a: 0000 add [eax], al
0000002c: 0000 add [eax], al
0000002e: 0000 add [eax], al
00000030: 0000 add [eax], al
00000032: 0000 add [eax], al
00000034: 0000 add [eax], al
00000036: 0000 add [eax], al
00000038: 0000 add [eax], al
0000003a: 0000 add [eax], al
0000003c: 0000 add [eax], al
0000003e: 0000 add [eax], al
00000040: 0000 add [eax], al
00000042: 0000 add [eax], al
00000044: 0000 add [eax], al
00000046: 0000 add [eax], al
00000048: 0000 add [eax], al
0000004a: 0000 add [eax], al
0000004c: 0000 add [eax], al
0000004e: 0000 add [eax], al
00000050: 0000 add [eax], al
00000052: 0000 add [eax], al
00000054: 0000 add [eax], al
00000056: 0000 add [eax], al
00000058: 0000 add [eax], al
0000005a: 0000 add [eax], al
0000005c: 0000 add [eax], al
0000005e: 0000 add [eax], al
00000060: 0000 add [eax], al
00000062: 0000 add [eax], al
00000064: 0000 add [eax], al
00000066: 0000 add [eax], al
00000068: 0000 add [eax], al
0000006a: 0000 add [eax], al
0000006c: 0000 add [eax], al
0000006e: 0000 add [eax], al
00000070: 0000 add [eax], al
00000072: 0000 add [eax], al
00000074: 0000 add [eax], al
00000076: 0000 add [eax], al
00000078: 0000 add [eax], al
0000007a: 0000 add [eax], al
0000007c: 0000 add [eax], al
0000007e: 0000 add [eax], al
00000080: 0000 add [eax], al
00000082: 0000 add [eax], al
00000084: 0000 add [eax], al
00000086: 0000 add [eax], al
00000088: 0000 add [eax], al
0000008a: 0000 add [eax], al
0000008c: 0000 add [eax], al
0000008e: 0000 add [eax], al
00000090: 0000 add [eax], al
00000092: 0000 add [eax], al
00000094: 0000 add [eax], al
00000096: 0000 add [eax], al
00000098: 0000 add [eax], al
0000009a: 0000 add [eax], al
0000009c: 0000 add [eax], al
0000009e: 0000 add [eax], al
000000a0: 0000 add [eax], al
000000a2: 0000 add [eax], al
000000a4: 0000 add [eax], al
000000a6: 0000 add [eax], al
000000a8: 0000 add [eax], al
000000aa: 0000 add [eax], al
000000ac: 0000 add [eax], al
000000ae: 0000 add [eax], al
000000b0: 0000 add [eax], al
000000b2: 0000 add [eax], al
000000b4: 0000 add [eax], al
000000b6: 0000 add [eax], al
000000b8: 0000 add [eax], al
000000ba: 0000 add [eax], al
000000bc: 0000 add [eax], al
000000be: 0000 add [eax], al
000000c0: 0000 add [eax], al
000000c2: 0000 add [eax], al
000000c4: 0000 add [eax], al
000000c6: 0000 add [eax], al
000000c8: 0000 add [eax], al
000000ca: 0000 add [eax], al
000000cc: 0000 add [eax], al
000000ce: 0000 add [eax], al
000000d0: 0000 add [eax], al
000000d2: 0000 add [eax], al
000000d4: 0000 add [eax], al
000000d6: 0000 add [eax], al
000000d8: 0000 add [eax], al
000000da: 0000 add [eax], al
000000dc: 0000 add [eax], al
000000de: 0000 add [eax], al
000000e0: 0000 add [eax], al
000000e2: 0000 add [eax], al
000000e4: 0000 add [eax], al
000000e6: 0000 add [eax], al
000000e8: 0000 add [eax], al
000000ea: 0000 add [eax], al
000000ec: 0000 add [eax], al
000000ee: 0000 add [eax], al
000000f0: 0000 add [eax], al
000000f2: 0000 add [eax], al
000000f4: 0000 add [eax], al
000000f6: 0000 add [eax], al
000000f8: 0000 add [eax], al
000000fa: 0000 add [eax], al
000000fc: 0000 add [eax], al
000000fe: 0000 add [eax], al

Built on Aegis on Fri Dec 24 13:18:44 2010 using compiler version 1400

Windows 6.1 (Windows Vista x64 build 7601) [Service Pack 1]

EAX = 05e67f78
EBX = 0280fce8
ECX = 05e67f04
EDX = 05e67f78
EBP = 0018f67c
ESI = 0280e898
EDI = 0280e89c
ESP = 0018f61c
EIP = 00000000
EFLAGS = 00210202
FPUCW = 027f
FPUTW = ffff

Crash reason: Access Violation

Crash context:
An out-of-bounds memory access (access violation) occurred in module 'VirtualDub'...

...reading address 00000000.

Pointer dumps:

EAX 05e67f78: 705c3a4b 72676f72 5c736d61 68747970 765f6e6f 2e322e33 65745c33 2e317473
EBX 0280fce8: 005fab98 00000001 00000000 0280e880 0280e880 00000000 00000000 00000100
ECX 05e67f00: 00000002 05e67f78 00000000 00000000 00000000 00000023 0000002f 00000000
EDX 05e67f78: 705c3a4b 72676f72 5c736d61 68747970 765f6e6f 2e322e33 65745c33 2e317473
ESI 0280e898: 005fa240 05e67efc 05e67ef0 00000001 17ad34de 80000000 00000056 00000000
EDI 0280e898: 005fa240 05e67efc 05e67ef0 00000001 17ad34de 80000000 00000056 00000000
ESP 0018f618: 05e67f04 6bff1e18 05e67f78 05e67f20 0280e89c 0280e898 0280fce8 0280fce8
0018f638: 0018f694 0053000e 656853a3 6bff7188 0018fbb0 00000000 00000000 0050000f
0018f658: 656853a3 00000000 04d3e980 027b0358 759f9e62 0018f6c8 0018f6cc 6bff18a4
0018f678: 05e67ef0 0018f6d8 6bff1dac 0280e89c 0280e898 0280fce8 0018f624 0018f6fc
EBP 0018f678: 05e67ef0 0018f6d8 6bff1dac 0280e89c 0280e898 0280fce8 0018f624 0018f6fc
0018f698: 0018f764 772471d5 0041f7e0 fffffffe 771fe38c 771fe0f2 00000010 00000000
0018f6b8: 771fe046 00730000 04d3e980 0018f72c 00000002 0018f728 6bff186a 05e67ef0
0018f6d8: 0018f72c 6bff2444 0280e89c 0280e898 0280fce8 0018f704 771fe023 028f8f00

Thread call stack:
6bff1e18: vsvfw!DllCanUnloadNow [6bff0000+14a0+978]
759f9e62: ole32!CoCreateInstanceEx [759b0000+49d4e+114]
6bff18a4: vsvfw!DllCanUnloadNow [6bff0000+14a0+404]
6bff1dac: vsvfw!DllCanUnloadNow [6bff0000+14a0+90c]
771fe38c: ntdll!RtlInitUnicodeString [771d0000+2e228+164]
771fe0f2: ntdll!RtlAllocateHeap [771d0000+2e046+ac]
6bff186a: vsvfw!DllCanUnloadNow [6bff0000+14a0+3ca]
6bff2444: vsvfw!DllCanUnloadNow [6bff0000+14a0+fa4]
771fe023: ntdll!RtlFreeHeap [771d0000+2dfa5+7e]
771fe023: ntdll!RtlFreeHeap [771d0000+2dfa5+7e]
005b2fb6: malloc()
004c74c4: AVIReadHandlerTunnelW32::AVIReadHandlerTunnelW32()
004c76ab: CreateAVIReadHandler()
004ce98b: InputFileAVI::Init()
772035a7: ntdll!RtlImageNtHeader [771d0000+33184+423]
7720389a: ntdll!RtlImageNtHeader [771d0000+33184+716]
75738443: USER32!GetClassLongW [75720000+183a8+9b]
772038be: ntdll!RtlImageNtHeader [771d0000+33184+73a]
7720389a: ntdll!RtlImageNtHeader [771d0000+33184+716]
7573d184: USER32!GetWindowLongA [75720000+1d156+2e]
7573d190: USER32!GetWindowLongA [75720000+1d156+3a]
7573d190: USER32!GetWindowLongA [75720000+1d156+3a]
77203492: ntdll!RtlImageNtHeader [771d0000+33184+30e]
7573d190: USER32!GetWindowLongA [75720000+1d156+3a]
7575f962: USER32!GetCursor [75720000+3f6e0+282]
7720389a: ntdll!RtlImageNtHeader [771d0000+33184+716]
77203492: ntdll!RtlImageNtHeader [771d0000+33184+30e]
7573717a: USER32!IsWindow [75720000+17136+44]
7573692a: USER32!gapfnScSendMessage [75720000+15fc8+962]
75737290: USER32!GetPropW [75720000+17227+69]
75737276: USER32!GetPropW [75720000+17227+4f]
75737257: USER32!GetPropW [75720000+17227+30]
75738e0d: USER32!RemovePropW [75720000+18dbd+50]
75738de4: USER32!RemovePropW [75720000+18dbd+27]
6e6876a5: uxtheme!GetThemeBool [6e670000+16651+1054]
6e687650: uxtheme!GetThemeBool [6e670000+16651+fff]
757362fa: USER32!gapfnScSendMessage [75720000+15fc8+332]
75736d91: USER32!GetThreadDesktop [75720000+16c63+12e]
75736d51: USER32!GetThreadDesktop [75720000+16c63+ee]
75736ce9: USER32!GetThreadDesktop [75720000+16c63+86]
75736d91: USER32!GetThreadDesktop [75720000+16c63+12e]
75736d51: USER32!GetThreadDesktop [75720000+16c63+ee]
75736de8: USER32!GetThreadDesktop [75720000+16c63+185]
75736df3: USER32!GetThreadDesktop [75720000+16c63+190]
75736df3: USER32!GetThreadDesktop [75720000+16c63+190]
75739a21: USER32!FindWindowW [75720000+198fd+124]
771ef8ca: ntdll!NtCallbackReturn [771d0000+1f8b8+12]
75739a49: USER32!FindWindowW [75720000+198fd+14c]
771e010a: ntdll!KiUserCallbackDispatcher [771d0000+100dc+2e]
7573692a: USER32!gapfnScSendMessage [75720000+15fc8+962]
7575cdb3: USER32!DialogBoxIndirectParamW [75720000+3cbf3+1c0]
7575cf5c: USER32!DialogBoxIndirectParamAorW [75720000+3ce54+108]
7577cb58: USER32!DialogBoxParamA [75720000+5cb0c+4c]
004cc7d9: InputFileAVI::promptForOptions()
00464e4e: VDProject::Open()
0051d29e: VDAutoLogger::VDAutoLogger()
00453e5c: OpenAVI()
0046c491: VDProjectUI::MenuHit()
75736b00: USER32!gapfnScSendMessage [75720000+15fc8+b38]
004794a0: VDUIFrame::DefProc()
0046da41: _catch$?MainWndProc@VDProjectUI@@IAEJIIJ@Z$0()
00465ac0: VDProjectUI::WndProc()
00479b65: VDUIFrame::StaticWndProc()
757362fa: USER32!gapfnScSendMessage [75720000+15fc8+332]
75736d3a: USER32!GetThreadDesktop [75720000+16c63+d7]
75736ce9: USER32!GetThreadDesktop [75720000+16c63+86]
771e010a: ntdll!KiUserCallbackDispatcher [771d0000+100dc+2e]
757377c4: USER32!CharPrevW [75720000+1768c+138]
757460dc: USER32!PeekMessageA [75720000+25f74+168]
75737bca: USER32!DispatchMessageA [75720000+17bbb+f]
00452596: WinMain@16()
005b78db: __tmainCRTStartup()
74b4339a: kernel32!BaseThreadInitThunk [74b30000+13388+12]
77209ef2: ntdll!RtlInitializeExceptionChain [771d0000+39e8f+63]
77209ec5: ntdll!RtlInitializeExceptionChain [771d0000+39e8f+36]

-- End of report

[Chikuzen]

I've reproduced it. But I'm not quite sure why it happens. I also redid the comparison with avs2yuv (to add the piping overhead in both cases) and avisynth is then still 50% faster.

Isn't it based on the difference in Bitblt() of avisynth and vapoursynth?
avisynth has ISSE optimized one, but ffms2 for VS has only pure C++ code.

[Myrsloik]

Isn't it based on the difference in Bitblt() of avisynth and vapoursynth?
avisynth has ISSE optimized one, but ffms2 for VS has only pure C++ code.

The one in VS is sse2 optimized. I disassembled msvcr100.dll myself and took a look. Did you really think microsoft wouldn't optimize such a common function as memcpy?

Oh, and the output is never explicitly copied in that way. It's passed to the WriteFile()

[lansing]

Nice, I have VFW working with MPC & AVISource, but I get access violations with VirtualDUB v1.9.11.
I've tried the newer beta vdub v1.10.2, but got same problem.

Any idea what could be wrong ?
Thanks in advance !



have you try changing the vpy's encoding to ANSI like i mention a few posts back? Did your script contain non English letter? Try make it all in English if that's the case.

Have you follow the instruction exactly the R10 update blog said? Like switching the "file of type" first before opening the vpy file?

http://www.vapoursynth.com/


if you got it to work with the avisource avs wrapper, it shouldn't be a problem opening the original vpy file with virtualdub.

[Pat357]

if you got it to work with the avisource avs wrapper, it shouldn't be a problem opening the original vpy file with virtualdub.
Got it working now ! Thanks a lot !
Don't know what exactly caused the problems : I deleted all registry settings from vdub, restarted PC and voila, it worked.
The codepage settings is a good tip to check if things all sudden "break" again.

[Keiyakusha]

Not sure if I did it right. Does everything really as good as it seems? Well maybe not that good, but It seems there is some improvements over pure avisynth.
so i made this script:
import vapoursynth as vs
import sys
core = vs.Core(threads=4)
core.std.LoadPlugin(path=r"..\ffms2.dll")
core.avs.LoadPlugin(path=r"..\mvtools2.dll")
ret=core.ffms2.Source(r"..\test8.mkv")
src=ret
c = core.avs.MSuper(c1=src,pel=2, sharp=1)
bv3 = core.avs.MAnalyse(c1=c, isb = 1, delta = 3, overlap=4)
bv2 = core.avs.MAnalyse(c1=c, isb = 1, delta = 2, overlap=4)
bv1 = core.avs.MAnalyse(c1=c, isb = 1, delta = 1, overlap=4)
fv1 = core.avs.MAnalyse(c1=c, isb = 0, delta = 1, overlap=4)
fv2 = core.avs.MAnalyse(c1=c, isb = 0, delta = 2, overlap=4)
fv3 = core.avs.MAnalyse(c1=c, isb = 0, delta = 3, overlap=4)
ret = core.avs.MDegrain3(c1=src,c2=c,c3=bv1,c4=fv1,c5=bv2,c6=fv2,c7=bv3,c8=fv3,thSAD=400)
last=ret

test clip is an SD 480p 8bit 24fps video
1) I opened this in avisynth via vfw module and checked speed. it was around 20.8fps, cpu load 30%
2) I ran the same script but in pure avs environment, speed was around ........ 8.77fps, cpu load 13%
3) Same as #2, but I enabled avisynth's multithreading with 4 threads. Speed: 27.8fps, cpu load 52%
3.5) Same as #3, but FFMS2 limited to 1 thread......................................... 30fps, cpu load 52%
4) Same as #2, but I enabled avisynth's multithreading with 8 threads. Speed: 36.8fps, cpu load 100%
5) doing the same as #1 but setting vs Core to 8 threads results.................. 21.6fps, cpu load 35% with spikes up to 52%

Now I need to chose which one is best compromise between speed/resources used ^__^
P.S.
I used mvtools from SVP and latest not broken SET's avisynth, measured with avsmeter 1.20

[Pat357]

1) I opened this in avisynth via vfw module and checked speed. it was around 20.8fps, cpu load 30%
3.5) Same as #3, but FFMS2 limited to 1 thread......................................... 30fps, cpu load 52%

It seems (1) and (3.5) are close to each other if you look at fps / %cpu (efficiency).

5) doing the same as #1 but setting vs Core to 8 threads results.................. 21.6fps, cpu load 35% with spikes up to 52%
Not exactly the best scaling I've ever seen :p
Something must be wrong : I can't believe the scaling is *that* bad !

[Myrsloik]

It seems (1) and (3.5) are close to each other if you look at fps / %cpu (efficiency).

Not exactly the best scaling I've ever seen :p
Something must be wrong : I can't believe the scaling is *that* bad !

The scaling isn't that bad. It's the vfw module that is. Unlike clip.output() it doesn't have multiple output frames going at once (the output function by default is doing 10 frames at once to keep throughput up).

Redo the test with script.py > NUL and it should reach near 100% cpu usage. Also, this isn't ricersynth, setting more threads than cores in vs most likely won't do you any good.

[Keiyakusha]

It seems (1) and (3.5) are close to each other if you look at fps / %cpu (efficiency).
Yep. It looks like so. But if in #1 case vapoursynth's processing uses totally separate threads from avisynth's one and then just supplies avisynth with frames, then by adding some more avisynth filters it may be more efficient than going pure AVS path. And my processing never limited to MDegrain only. I also didn't tried limiting ffms2 to 1 thread in vapoursynth's case, it may add few frames per second too. Edit: but this may be not an option for HD video, because 1 thread won't be fast enough to prepare decoded frames...

It seems (1) and (3.5) are close to each other if you look at fps / %cpu (efficiency).
its not that bad as it looks. see avisynth's #4, 50% cpu is capable of doing 30fps (tested cpu is 8 cores btw), but 100% only ~37fps? where all those resources are going!? I wonder if SET is hiding something like SETI@home in his builds ^__^
at least vapoursynth doesn't eats CPU that much if it can't make use of it.

[06_taro]

Test some clips, avsr/avs/native:

vs-ffms-avsr.py
#!/bin/env python
# coding: utf-8

import vapoursynth as vs
import sys

core = vs.Core()

core.std.LoadPlugin(path=r"d:\Program Files\Media\VapourSynth\vsavsreader.dll")
ret = core.avsr.Eval(r"""FFVideoSource("e:\test_clip-1280x720_8bpp.mp4")""")

ret.output(sys.stdout, y4m=True)

vs-ffms-avs.py
#!/bin/env python
# coding: utf-8

import vapoursynth as vs
import sys

core = vs.Core()

core.avs.LoadPlugin(path=r"d:\Program Files\Media\VapourSynth\ffms2.dll")
ret = core.avs.FFVideoSource(r"e:\test_clip-1280x720_8bpp.mp4")

ret.output(sys.stdout, y4m=True)

vs-ffms-native.py
#!/bin/env python
# coding: utf-8

import vapoursynth as vs
import sys

core = vs.Core()

core.std.LoadPlugin(path=r"d:\Program Files\Media\VapourSynth\ffms2.dll")
ret = core.ffms2.Source(r"e:\test_clip-1280x720_8bpp.mp4")

ret.output(sys.stdout, y4m=True)

At first I found avsr was slightly faster (really slightly, no more than 2%), meaning totally use avs runtime wrapper was faster, but I was aware that ffms2.dll in my avs plugin directory was an old version r700 compiled by SAPikachu. Replaced it by latest r725 icl version (didn't find a gcc+msvc version) and re-tested, avsr version was the slowest of all, but the difference was not that large as in some posts in the last pages:
http://www.nmm-hd.org/upload/get~JDDMbfbPmBg/avsr-avs-native.png

Test clip was 1280x720 8-bit avc in mp4, avisynth.dll was 2.60 MT 28 Aug by SEt, and VapourSynth was r10. CPU usage was about the same, ~87% on my i7-2670qm, as I had set the affinity of bash and its child processes to 0x7F and all other CPU-comsuming processes in the background ( mainly browsers ) to 0x80.

Also tested on some other 8-bit sources, and in most cases the results were similar.

[AzraelNewtype]

There is no good lossless formats for 8+ bitdepth though... At least not the ones that are free and supported by different software

I use lossless hi10p h.264 weekly, x264 has no qualms about generating it, and ffms2 can read it back in. In fact, in vs, it will read it back at native 10-bit rather than dithering to 8-bit like the avs version of the function. It's Neat.

[Myrsloik]

I've released r11. Update to Python 3.3 before installing. More or less completes the VFW stuff and fixes the performance issues. Redo the VFW benchmarks with this version and see what happens...

Here's a blog post as usual.
R11 – VFW returns and Python 3.3 (http://www.vapoursynth.com/2012/10/r11-vfw-returns-and-python-3-3/)

[kolak]

What are the 8bit+ formats supported- P210?

[Myrsloik]

P010, P016, P210, P216 are the ones I added. I tested with mpc-hc and madvr. I didn't add any 444 formats because the ones specified by ms are packed only.
As usual pushing to have other programs support these formats is appreciated.

[kolak]

v210, ProRes seams to be fine and madVR reports P210 :) I hope for better support for these formats.

What do I have to type in resizer format to scale in P210 format?

[Myrsloik]

v210, ProRes seams to be fine and madVR reports P210 :) I hope for better support for these formats.

What do I have to use to scale in P210 format?

To scale in? I don't understand what you mean. The format constants are YUV420P10/16 and YUV422P10/16 for use with the resizers if that's what you mean.

[kolak]

Sorry- yes in resizer :)

It works, but I don't have anything to convert P210 format to eg v210, to be useful :)
Need to wait for some solutions :(

[lansing]

forget about this benchmarks because I messed up output big time

[pie]

I'm not sure if I'm doing something wrong, but whenever I try to feed a 10-bit h264 clip as input (using ffms2) (P010 format?) I'm getting 'Frame returned not of the declared type' whenever I try to output it.

Edit:
Ok my bad - was passing it to a filter first which probably doesn't even support 10-bit input.

[Keiyakusha]

I use lossless hi10p h.264 weekly, x264 has no qualms about generating it, and ffms2 can read it back in. In fact, in vs, it will read it back at native 10-bit rather than dithering to 8-bit like the avs version of the function. It's Neat.

unfortunately professional editing software not allows to save output in h264 10-bit lossless, and I need format that can contain 16bit(or 32) information more often than 10bit.
Edit: actually this is not a problem. I was just saying that ability to process this kind of images in vapoursynth will be good. Since it doesn't have bitdepth limitations.

[JEEB]

Sorry- yes in resizer :)

It works, but I don't have anything to convert P210 format to eg v210, to be useful :)
Need to wait for some solutions :(
As I said, if you want v210 you go and push out the raw yuv420p10 into ffmpeg or avconv, and then output v210. Vapoursynth python scripts output their stuff to stdout so it has already been possible for quite a while.

v210 is not an in-memory colorspace that you want to use for moving of pictures in memory for other filters/applications, as it starts packing values into the "leftover" bits of the used bytes to save space. Which makes sense for file output (get a somewhat smaller file size), but makes exactly zero sense as something that would get used deliver pictures from within memory IMHO.

[active1]

i have an error on ubuntu after compiling vapoursynth

i compiled it successfully with the install instruction and i get this error after:
import vapoursynth as vs
import sys
and when:
c = vs.Core()

AttributeError: 'module' object has no attribute 'Core'

python version is 3.2.2
NOTE: i tried that before 24 hours, so vapoursynth was in r10

vfw now has support for the P010, P016, P210, P216 formats, tested in mpc-hc with madvr
great!! :thanks:

[Myrsloik]

i have an error on ubuntu after compiling vapoursynth

i compiled it successfully with the install instruction and i get this error after:
import vapoursynth as vs
import sys
and when:
c = vs.Core()

AttributeError: 'module' object has no attribute 'Core'

python version is 3.2.2
NOTE: i tried that before 24 hours, so vapoursynth was in r10


great!! :thanks:

I'm not sure why it won't work for you. Did you use a recent cython version? Does print(vs.YUV420P8) work?

My sources tell me that this happened with some revisions. Try updating to the latest revision.

[kolak]

As I said, if you want v210 you go and push out the raw yuv420p10 into ffmpeg or avconv, and then output v210. Vapoursynth python scripts output their stuff to stdout so it has already been possible for quite a while.

v210 is not an in-memory colorspace that you want to use for moving of pictures in memory for other filters/applications, as it starts packing values into the "leftover" bits of the used bytes to save space. Which makes sense for file output (get a somewhat smaller file size), but makes exactly zero sense as something that would get used deliver pictures from within memory IMHO.

Understand this, that's why I said it can be implemented on final level/output in some tool like avfs. Can this be done?

[JEEB]

Understand this, that's why I said it can be implemented on final level/output in some tool like avfs. Can this be done?
Yes? As far as I know you can get raw 10bit YCbCr that ffmpeg/avconv understands from Vapoursynth, and you can output v210 from it. See a part of the post you yourself quoted.
As I said, if you want v210 you go and push out the raw yuv420p10 into ffmpeg or avconv, and then output v210. Vapoursynth python scripts output their stuff to stdout so it has already been possible for quite a while.

it would be something like ffmpeg/avconv -f rawvideo -s WIDTHxHEIGHT -pix_fmt yuv422p10le -i - -c:v v210 out.mov
from the ffmpeg/avconv side. Just pipe yuv422p10le or the be alternative from vapoursynth (and switch the pix_fmt as needed). I am assuming that a lone - will work as the marker for "input comes from stdin".

[Myrsloik]

Now I need some input on which frame statistics to implement. If you generalize all of the functions in avisynth you get these 6 left. I've seen that difference is used a lot but are there any of these I could skip? I can't remember seeing the min/max/median used ever.
You can also suggest new ones if they're not too complicated.

Average(clip)
Difference(clip1, clip2)
PlaneMax(clip, float threshold)
PlaneMin(clip, float threshold)
PlaneMedian(clip)
PlaneMinMaxDifference(clip, float threshold)

The full list here:
http://avisynth.org/mediawiki/ScriptClip

[Keiyakusha]

Is it possible to mimic histogram(mode=luma) function of avisynth? So the output will be exactly the same? I wonder how that will look with 8+ bit streams... For me this is probably most often used function in avisynth, not counting ffms2 as source filter.
Well, actually there can be one difference. avisynth's histogram mode-luma have a bug/feature, while keeping colorspace intact it expands tv levels to PC and user have no control over that. Also I don't see why it should be limited to luma only. Specifying which plane you want to see as an option should be not bad idea.

[kolak]

Yes? As far as I know you can get raw 10bit YCbCr that ffmpeg/avconv understands from Vapoursynth, and you can output v210 from it. See a part of the post you yourself quoted.


it would be something like ffmpeg/avconv -f rawvideo -s WIDTHxHEIGHT -pix_fmt yuv422p10le -i - -c:v v210 out.mov
from the ffmpeg/avconv side. Just pipe yuv422p10le or the be alternative from vapoursynth (and switch the pix_fmt as needed). I am assuming that a lone - will work as the marker for "input comes from stdin".

Thanks, but this is not my goal. I use ffmpeg, when needed to create v210, but I don't want big file :)
What I want is to turn script into fake AVI v210 file, to avoid rendering time and save space. I want exactly the same as avfs does, but with v210 format at the output as this would make it compatible with every software I use. My whole v210 idea is just for broad compatibility with many different software. As far as I understand you have to implement v210 format into avfs. AVFS author was not interested in it, as before there was no proper 10bit format in avisynth. Now it has changed, so I hope such a tool will be done. I can donate money for it, no problem. At the end it's not that difficult is it (I'm not a programmer, so can't judge)?

[Andouille]

Is it possible to mimic histogram(mode=luma) function of avisynth? So the output will be exactly the same? I wonder how that will look with 8+ bit streams... For me this is probably most often used function in avisynth, not counting ffms2 as source filter.
Well, actually there can be one difference. avisynth's histogram mode-luma have a bug/feature, while keeping colorspace intact it expands tv levels to PC and user have no control over that. Also I don't see why it should be limited to luma only. Specifying which plane you want to see as an option should be not bad idea.

The equivalent version with masktools is near 3 times faster

mt_lut (expr="x 16 &u 17 x 15 &u - 1 x 15 &u + ? 14 *", u=-128, v=-128)

edit: assuming there is a vapoursynth version of masktools

[Myrsloik]

The equivalent version with masktools is near 3 times faster

mt_lut (expr="x 16 &u 17 x 15 &u - 1 x 15 &u + ? 14 *", u=-128, v=-128)

edit: assuming there is a vapoursynth version of masktools

See the documentation for std.lut (and lut2).

[JEEB]

What I want is to turn script into fake AVI v210 file, to avoid rendering time and save space. I want exactly the same as avfs does, but with v210 format at the output as this would make it compatible with every software I use. My whole v210 idea is just for broad compatibility with many different software.
Tell me software that takes in v210 in AVI. As far as I know, that pretty much limits itself to VirtualDub. It is only standard in Quicktime's media format (mov) as far as I know.

I would be happy to be proven otherwise though, but I'm not exactly counting on it :P

Edit: Take in as in actually in 10bit 4:2:2. There are VFW components from "Drastic.tv" and another from something called "AJA". I am very much unsure if those will actually give 10bit YCbCr to most applications in, in which case whether or not it's 10bit loses all meaning. Not to mention that the V210 used by those might be a completely different thing compared to the one used in Quicktime. But as I said, you may surprise me if you choose so.

[Keiyakusha]

Tell me software that takes in v210 in AVI.
pretty much any professional software? AfterEffects would be good example. It happily creates and takes it.

edit: believe it or not, even photoshop takes it. you need extended version to enable video editing

[Reel.Deel]

I can confirm Sony Vegas accepts and creates avi V210.

Avi created with Vegas.
Format : YUV
Codec ID : v210
Codec ID/Hint : AJA Video Systems Xena
Color space : YUV
Chroma subsampling : 4:2:2
Bit depth : 10 bits
Compression mode : Lossless

[JEEB]

pretty much any professional software? AfterEffects would be good example. It happily creates and takes it.

Is that really in AVI?
Is that really the same type of V210 that is used in MOV? Or is it something else?
Is it really natively supported, or is it random VFW modules that are being used, as this usually makes it harder to believe that the application is actually giving out 10bit YCbCr in there instead of the VFW encoder encoding whatever it is given into 10bit YCbCr (which could be OK if it is the one doing the possible RGB->YCbCr conversion and it is doing it well)? As for decoding, the same things can be asked about how and in what sense the stuff gets loaded in.


I'm sorry for being a possible arse, but I just know way too well that the truth inside certain "pro" workflows aren't as good as they are made to sound. Using something like the V210 from Apple only makes sense to get hacked somewhere if it really, really is supported in VFW/AVI properly. Otherwise it really has zero reason to exist anywhere close to any output format for something like VapourSynth.

Edit:
Just heard that in most cases the VFW V210 "decoders" available in most cases output RGB to the calling application. How is this better than just putting out RGB to begin with, and do some !VirtualDub applications actually support the format as actual 10bit YCbCr?

[Pat357]

Where can I get "VSavsreader.dll" from Chikuzen ?

[StainlessS]

VSavsreader by Chikuzen
http://forum.doom9.org/showthread.php?t=165957

[Keiyakusha]

Is that really in AVI?
Is that really the same type of V210 that is used in MOV? Or is it something else?
Is it really natively supported, or is it random VFW modules that are being used, as this usually makes it harder to believe that the application is actually giving out 10bit YCbCr in there instead of the VFW encoder encoding whatever it is given into 10bit YCbCr (which could be OK if it is the one doing the possible RGB->YCbCr conversion and it is doing it well)? As for decoding, the same things can be asked about how and in what sense the stuff gets loaded in.

I think mediainfo answers 1st 2 questions.
3rd not sure if I fuly understand you, but it doesn't depends on anything in windows to be able to read or write that

this it creates and reads edit: it is not converted to anything, I'm getting real 10bit stuff when open this file. this is why they made it to begin with.
Format : AVI
Format/Info : Audio Video Interleave
File size : 63.3 MiB
Duration : 501ms
Overall bit rate : 1 060 Mbps
Writing application : Adobe After Effects CS6 (Windows)

Video
ID : 0
Format : YUV
Codec ID : v210
Codec ID/Hint : AJA Video Systems Xena
Duration : 501ms
Bit rate : 1 061 Mbps
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate : 23.976 fps
Color space : YUV
Chroma subsampling : 4:2:2
Bit depth : 10 bits
Compression mode : Lossless
Bits/(Pixel*Frame) : 21.333
Stream size : 63.3 MiB (100%)

Format : MPEG-4
Format profile : QuickTime
Codec ID : qt
File size : 63.3 MiB
Duration : 501ms
Overall bit rate mode : Constant
Overall bit rate : 1 060 Mbps
Writing library : Apple QuickTime
ｩTIM : 00:00:00:00
ｩTSC : 2997
ｩTSZ : 125

Video
ID : 1
Format : YUV
Codec ID : v210
Codec ID/Hint : AJA Video Systems Xena
Duration : 501ms
Bit rate mode : Constant
Bit rate : 1 061 Mbps
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 23.976 fps
Color space : YUV
Chroma subsampling : 4:2:2
Bit depth : 10 bits
Compression mode : Lossless
Bits/(Pixel*Frame) : 21.333
Stream size : 63.3 MiB (100%)

[JEEB]

3rd not sure if I fuly understand you, but it doesn't depends on anything in windows to be able to read or write that
That indeed does answer most of my questions, but the final one is actually the most important. Does this enable you really, really to get 10bit YCbCr into those applications as such, like in theory you are getting on the QT side?

If yes, then it kind of makes sense. Adding this overcomplicated format that can barely be called "raw" (you can't just call out a certain pixel without doing bit-based parsing) just to have you get YUY2 or RGB out of it really makes no sense, and that is why I have been so adamant on the actual users of this format (mainly kolak, who seemingly is rather out-of-touch with the technical issues) to be sure that they are actually getting 10bit YCbCr out of it.

Edit: Meh, if someone wants to implement it its their own thing. I just seemingly got tired of someone poking/spamming about it all the time while it was probably better off being implemented in something that possibly isn't VapourSynth itself, but something that possibly uses the API or the output from VapourSynth.

[lansing]

I wanted to load a avs core function like TemporalSoften, since it doesn't have it's own dll, how do i call it in vpy?

[ajp_anton]

Now I need some input on which frame statistics to implement. If you generalize all of the functions in avisynth you get these 6 left. I've seen that difference is used a lot but are there any of these I could skip? I can't remember seeing the min/max/median used ever.
You can also suggest new ones if they're not too complicated.

Average(clip)
Difference(clip1, clip2)
PlaneMax(clip, float threshold)
PlaneMin(clip, float threshold)
PlaneMedian(clip)
PlaneMinMaxDifference(clip, float threshold)

The full list here:
http://avisynth.org/mediawiki/ScriptClipIsn't PlaneMinMaxDifference just PlaneMax-PlaneMin? Haven't tried it, just going by the name.
Can't difference be done using lut2 and average?

I've used Max and Min a few times. Please don't go the Apple route and remove everything that's not absolutely crucial, leave something for people to play around with =).

Maybe you should name them all Plane[...], e.g. PlaneAverage etc. Then add an argument for choosing which luma/chroma/RGB plane to use.

[Keiyakusha]

That indeed does answer most of my questions, but the final one is actually the most important. Does this enable you really, really to get 10bit YCbCr into those applications as such, like in theory you are getting on the QT side?
Well, I wasn't following your discussion with kolak, sorry. I just spotted interesting question. Personally I don't use this format, I prefer something losslessly compressed. After importing, software reports that it have trillions of colors, but i never did any tests to check if this is really so. But I don't think they will implement this just to dither it silently at some point. Any formats that are supported and opened not via directshow are opened without conversions.

[Pat357]

Well, I wasn't following your discussion with kolak, sorry. I just spotted interesting question. Personally I don't use this format, I prefer something losslessly compressed. After importing, software reports that it have trillions of colors, but i never did any tests to check if this is really so. But I don't think they will implement this just to dither it silently at some point. Any formats that are supported and opened not via directshow are opened without conversions.
Isn't the Windows VfW engine used to open these files ?
What happens if you drop such a v210 file in MPC-HC/MadVR ?
What exact error do you get (if any) ?

[kolak]

Tell me software that takes in v210 in AVI. As far as I know, that pretty much limits itself to VirtualDub. It is only standard in Quicktime's media format (mov) as far as I know.

I would be happy to be proven otherwise though, but I'm not exactly counting on it :P

Edit: Take in as in actually in 10bit 4:2:2. There are VFW components from "Drastic.tv" and another from something called "AJA". I am very much unsure if those will actually give 10bit YCbCr to most applications in, in which case whether or not it's 10bit loses all meaning. Not to mention that the V210 used by those might be a completely different thing compared to the one used in Quicktime. But as I said, you may surprise me if you choose so.

Almost every editing software: Premiere, Vegas, Edius, FCP, Lightgworks, grading software: Resolve, Scratch, etc (through ref MOV), all capture cards/their software: AJA, BlackMagic, Blufish444 etc, most open source software: Vdub, ffmpeg....

It's actually hard to name software, which does not open v210 AVI or MOV. Whole broadcast industry is based on v210 (if I'm correct HD-SDI standard uses v210 format).
It's the same format for MOV and AVI and can be easily re-wrapped with eg ffmpeg.

[Myrsloik]

Isn't PlaneMinMaxDifference just PlaneMax-PlaneMin? Haven't tried it, just going by the name.
Can't difference be done using lut2 and average?

I've used Max and Min a few times. Please don't go the Apple route and remove everything that's not absolutely crucial, leave something for people to play around with =).

Maybe you should name them all Plane[...], e.g. PlaneAverage etc. Then add an argument for choosing which luma/chroma/RGB plane to use.

I'm not quite pulling an Apple, I just happen to be a single unemployed human being who codes between applying for jobs. That's why I try to spread the functions as much as possible, so if you can synthesize a function in 3 steps or less from the available ones then I won't implement it... for now.

That reminds me, I should try to extract some of the more useful avisynth core functions as a separate plugin.

[kolak]

Isn't the Windows VfW engine used to open these files ?
What happens if you drop such a v210 file in MPC-HC/MadVR ?
What exact error do you get (if any) ?

If you have 10bit VFW codec (Drastic, AJA, BM , LAV, Cineform etc) it gets decoded to some 8bit format (YUY2, RGB etc) and passed to be displayed/processed. This is when some software does not support 10bit natively.
With something like madVR it gets displayed at 10bit directly.
Most of the software will read 10bit data directly also, so you can process it at high precision. This is whole idea to have access to "real 10bit" from almost every software and something like fake AVI (like avfs does for 8bit) is great solution to save space and make things easy and simple :)

[kolak]

Well, I wasn't following your discussion with kolak, sorry. I just spotted interesting question. Personally I don't use this format, I prefer something losslessly compressed. After importing, software reports that it have trillions of colors, but i never did any tests to check if this is really so. But I don't think they will implement this just to dither it silently at some point. Any formats that are supported and opened not via directshow are opened without conversions.

Yes- this is an "other" solution and I use it with lav decoder and avfs :) Sometime I need real 10bit data thought.

[Keiyakusha]

Isn't the Windows VfW engine used to open these files ?
What happens if you drop such a v210 file in MPC-HC/MadVR ?
What exact error do you get (if any) ?

no. not vfw
under x86 architecture I can play this file using LAV source filter+madvr because madvr supports direct v210 input, this is the only way I know. http://i.imgur.com/eZT99.png (without installing 3rd party software that I don't have anyway)
under pure x64 I have no way to play this outside editing software. and adobe dropped x86 support anyway.

edit: oops I'm slow...

[JEEB]

...
I already finished my discussion on this with the last edit I had, but it seems like I have to really "bend my point for you with iron wire", as the point seems to have gone over your head in any case.

First of all, yes -- V210 in MOV is a standard. That can be expected to work as advertised in QuickTime and related applications that use similar ways of loading the material up. I never said anything about that, and I knew that it's a more or less standard way of doing it on the Mac side of things (and Windows-based solutions that can load up V210 in MOV). My point and questioning was all the time only, and only about V210 in AVI and related Windows-based solutions.

In other words, I was more or less trying to find out if anyone had actually checked if that 10bit YCbCr that is in V210 was actually getting loaded as such from AVI. No-one so far has really said anything towards this, and the damn VFW components available from two different companies regarding this in most cases output RGB or possibly 8bit YCbCr. I know checking this in closed applications isn't easy, but it should be possible.

I hope you understand my point better now.

Disregarding that, the nature of V210 is what it is, and you are better off asking for it in some other context and less spammy. Or just get someone implement it for you, shouldn't be too hard if you can find someone with the time, interest and capabilities. Just don't spam it all the time.

[kolak]

I still don't see a problem (your point)- v210 is a way of storing RAW data (just packed) and (I assume) it's not that difficult to read this data natively and MANY apps do it.
I don't know other 10bit format, which is as well supported maybe except DPX or other image sequences- do you know any?
As far as I know it's bit more difficult to store v210 in AVI, but in the same time v210 in AVI is also very well supported (as per my other post).
8bit output is only for apps which don't support 10bit natively- similar to fact that some software may not support YUY2, but does RGB, so it needs "converter/codec" to read this data.

Yes- there is enough info in this thread about v210.

[Keiyakusha]

By the way until now I didn't know LAV decoder is able to "decode" (convert) v210, just found out about it! But anyway this is turned off by default, and as I said, adobe software supports x64 only, so LAV x86 won't work anyway.
edit: also not sure if this is related, but after JEEB mentioned YUV-RGB conversions I think I should note that premiere for example tries not to do conversion where it is possible. So depending on processing done, we can have yuv in - yuv out with no conversion to rgb (well, only for display)

[kolak]

Use LAV decoder and map your script as fake YUY2 AVI with avsp and load this to Adobe apps :)
Add distributor for multithreading with avsp.
Sorry for spam, but can't send private message to Keiyakusha.

[JEEB]

I still don't see a problem (your point)
My point was whether or not that pro stuff actually reads it natively or not from AVI, because it really would be completely and utterly useless if it would then just get converted to RGB or 8bit YCbCr when being read up. I know you want it, but I wanted to make sure that you can actually reap the damn rewards from what you are asking.

OK? Simple enough? That was it. And the thing that your posts about V210 were getting annoying. It's handled as a video codec instead of a colorspace in ffmpeg/libav for a reason.

Also, looking at the answers of people who are seemingly less oblivious, and giving them the benefit of doubt, it would seem that at least some of those tools actually read V210 properly from AVI instead of raping it. So that's it.

[lansing]

this reminds me a lot of the dude in the pcsx2 emulator development page a year back, who spam every revision updates asking to fix his precious super robot taisen game and thumbing down every single revision he sees. And one of the developer got so pissed and this happened:

http://code.google.com/p/pcsx2/source/detail?r=4525

[kolak]

My point was whether or not that pro stuff actually reads it natively or not from AVI, because it really would be completely and utterly useless if it would then just get converted to RGB or 8bit YCbCr when being read up. I know you want it, but I wanted to make sure that you can actually reap the damn rewards from what you are asking....



No problem- I just jumped few posts back and to answer your question- ALL software which I mentioned reads REAL data out of v210 AVI. ONLY software, which does not support v210 natively will use codec (AJA, BM etc), which will output some 8bit data (in most cases conversion is without any dithering, so quite crap). MOV support is bit better, but if you have v210 AVI you can make reference MOV and it also will work as this is the same raw data:). I hope it's clear now, v210 case is closed and I hope they will be some "port" from vpy to fake v210 AVI (or MOV).

[Myrsloik]

Now you're giving me ideas :sly:

[kolak]

At least can help in this way :)

[active1]

I'm not sure why it won't work for you. Did you use a recent cython version? Does print(vs.YUV420P8) work?
i used Cython-0.17.1
and print(vs.YUV420P8) give me the same error:
AttributeError: 'module' object has no attribute 'YUV420P8'


My sources tell me that this happened with some revisions. Try updating to the latest revision.

ok, i will :)

[lansing]

how do you run test using the "$ time..." command? I try typing it in my cmd and it said $ is not regconized

[mastrboy]

how do you run test using the "$ time..." command? I try typing it in my cmd and it said $ is not regconized

It's not a builtin windows command, you can install cygwin to get most of the basic *nix commands for windows.
(And the $ is just the PS1 variable, just like prompt in windows cmd, so the command is just "time")

[Mug Funky]

I looked at v210 but it's packing made me nauseous just to look at. I suppose I'll add it sooner or later but it's a shame it turned out to be the most popular. Microsoft defines very sane and clean planar formats officially too. What a shame... over 8 bit output will come. But it's a bit lower in priority since to my knowledge x264 is the only thing that truly benefits from it. Correct me if I'm wrong.

hehe... hope the dream isn't gone.

i wouldn't dare add to the v210 spam without adding to your donation bucket (and i'm broke-arse at the moment), but i'll give you a little background on the way it's like that.

basically it goes down the pipes that way, and the standards came from a time when computers were big and dumb and storage was precious (and mostly tape-based). capture cards grab this v210 off the SDI cables as-is, demux the audio channels and the video, and stuff it all into whatever container the software wants.

quicktime itself will load v210 avi files no problem (as will qtinput in avisynth, which is a lifesaver)
blackmagic's capture program will only capture v210 in avi on PC's, but will do mov on mac
CCE 2 and 3 will take v210 avi files and do the right thing with them
sonic's blu-ray encoder takes it too, without issue

like i say, not trying to spam, not requesting a feature, but just offering some of my experience with the format. it is what it is because that's how it comes off the cables, and every facility is wired with the same cables unfortunately. when HD-SDI came in, they of course made it backward-compatible and used the same packing, even though they were in a position to make things a bit easier for developers :)

[lansing]

After a couple of days of testing, i'm pretty safe to say that right now vapoursynth is about 2.5x faster than the official Avisynth overall, with cpu usage range from 70-80%; and about 30% slower than avisynth mt.

The memory control was just very nice,

official avs: 300MB always
avs-mt: 580MB+ always
vapoursynth: < 130MB.

And for mdegrain1 blocksize=16, i even got 45MB.

[Myrsloik]

After a couple of days of testing, i'm pretty safe to say that right now vapoursynth is about 2.5x faster than the official Avisynth overall, with cpu usage range from 70-80%; and about 30% slower than avisynth mt.

The memory control was just very nice,

official avs: 300MB always
avs-mt: 580MB+ always
vapoursynth: < 130MB.

And for mdegrain1 blocksize=16, i even got 45MB.

Now for the obvious question. On how many cores? 4?

What if you set the number of threads to cores+1 in your computer?

[lansing]

Now for the obvious question. On how many cores? 4?

yes, AMD Athlon2 X4 635.
For official avs, only one core was working; vapoursynth, all 4 cores working in balance

update:

setting vs.Core(threads=cores+5) results vary between +-3% comparngs to threads=cores

update 2:
here's the benchmarks done with "time", clip 720x480, 5000 frames, all done on 4 cores:

[Keiyakusha]

Wanted to compare some mvtools versions. Same script as I posted before, FFMS2 1 thread, through vfw, VS r11:
mvtools-v2.5.11.3 vsCore_4_threads 19.28FPS ~46%CPU
mvtools-2.5.11.9-svp vsCore_4_threads 25.18FPS ~46%CPU
mvtools-v2.5.11.3 vsCore_9_threads 22.30FPS ~70%CPU
mvtools-2.5.11.9-svp vsCore_9_threads 29.68FPS ~70%CPU
mvtools-2.6.0.5-dither vsCore_9_threads if I try to open it in whatever way, host application just closes without saying anything, and pure python only writes this: "YUV4MPEG2 C420 W720 H480 F24000:1001 Ip A0:0", without saying anything ^__^
^ dither mt-true in this case
mvtools-2.6.0.5-dither vsCore_4_threads 18.44FPS ~46%CPU
mvtools-2.6.0.5-dither vsCore_9_threads 21.12FPS ~70%CPU
^ dither mt-false

Also singlethreaded FFMS2 is not a bottleneck. Adding more threads actually slows processing.

[lansing]

mvtools-2.6.0.5-dither vsCore_9_threads if I try to open it in whatever way, host application just closes without saying anything, and pure python only writes this: "YUV4MPEG2 C420 W720 H480 F24000:1001 Ip A0:0", without saying anything ^__^

for the dither one, you need to set mt=false on all manalyse, msuper, and mdegrain in order to work in vpy

[Myrsloik]

mvtools-2.6.0.5-dither vsCore_9_threads if I try to open it in whatever way, host application just closes without saying anything, and pure python only writes this: "YUV4MPEG2 C420 W720 H480 F24000:1001 Ip A0:0", without saying anything ^__^

That's odd, I actually used the dither mvtools for my testing here. After I finish my current request, v210 and vsfs, I'll try to figure out why vs isn't as fast as it could be.

Interesting benchmarks, keep them coming. They help me identify where to look to improve things.

[Keiyakusha]

for the dither one, you need to set mt=false on all manalyse, msuper, and mdegrain in order to work in vpy

Oh I see, had no idea this matters. Well I added dither version to above post
mvtools-2.6.0.5-dither vsCore_4_threads 18.44FPS ~46%CPU
mvtools-2.6.0.5-dither vsCore_9_threads 21.12FPS ~70%CPU

That's odd, I actually used the dither mvtools for my testing here. After I finish my current request, v210 and vsfs, I'll try to figure out why vs isn't as fast as it could be.

Interesting benchmarks, keep them coming. They help me identify where to look to improve things.
Sorry didn't know I need to use mt-false in dither version

[lansing]

just out of curiosity, does it matters to have a 64bit vapoursynth going along with a 64bit python? Will it be faster than 32bit vapoursynth + 32bit python right now? Or am I talking nonsense here?

[Myrsloik]

just out of curiosity, does it matters to have a 64bit vapoursynth going along with a 64bit python? Will it be faster than 32bit vapoursynth + 32bit python right now? Or am I talking nonsense here?

It will be exactly the same with the existing ported filters. Except that you can't use any avisynth filters. A 64bit windows version is scheduled for when the plugins and other things are ready for it.

It does however run perfectly well in 64bit on linux if you really need those extra pointless bits RIGHT NOW(tm)

[Reel.Deel]

Hello, I was trying to do some speed test with nnedi3 but I encountered something different.

Only when I use ffms2 followed by nnedi3, the frames return out of order in both stdout and vfw. I loaded ffms2 as std and avs and I still had the same problem.
Strangely if I load the video with ffms2 alone and save it losslessly (ut video) and load it back up with ffms2 followed by nnedi3, I get no problem. :confused:
I tried multiple source filters like DGDec, DGDecNV, VsAvsReader and they all work fine with nnedi3. ffms2 followed by nnedi3 works fine in Avisynth.

The test clip is from a NTSC DVD. Here's the clip. (http://www.mediafire.com/?zv5hxduox4hh6ld)
Here's the output (http://www.mediafire.com/?019swxqp4idp4hc) with the frames out of order.

I used both ffms2 r725 icl (http://forum.doom9.org/showthread.php?p=1592858#post1592858) and icl 2 (http://forum.doom9.org/showthread.php?p=1593190#post1593190). Same problem.

import vapoursynth as vs
import sys
core = vs.Core(accept_lowercase=True, threads=4)

# Load Filters
core.std.LoadPlugin(path=r'C:\Vapoursynth\FFMS2\ffms2.dll')
#core.avs.LoadPlugin(path=r'C:\Vapoursynth\FFMS2\ffms2.dll')
core.avs.LoadPlugin(path=r'C:\Vapoursynth\NNEDI3\nnedi3.dll')

# Source
src = core.ffms2.source(source=r'C:\VapourSynth\test\test.m2v',threads=1)
#src = core.avs.FFVideoSource(source=r'C:\VapourSynth\test\test.m2v',threads=1)

# Proccesing
src = core.avs.nnedi3(c1=src,field=-2,nsize=6,nns=2)

# Ouput for VDub
last = src

# Output for pipe
#last.output(sys.stdout, y4m=True)

[Are_]

ffms has problems with certain containers, and the slower the filter chain the problem becomes worse.
Can you test if remuxing into mkv solves it?

[Reel.Deel]

ffms has problems with certain containers, and the slower the filter chain the problem becomes worse.
Can you test if remuxing into mkv solves it?

Yes that indeed solve it. Should of known better. :o
The odd thing is that it works fine in Avisynth without having to remux into mkv. :confused:

[Pat357]

Wanted to compare some mvtools versions. Same script as I posted before, FFMS2 1 thread, through vfw, VS r11:
How did you limit FFMS2 to 1 thread ?
I've tried this :
core.std.LoadPlugin(path=r"k:\programs\ffms2-r725-icl\ffms2.dll", threads=1)
but I got the (atm not yet, but will soon be) famous 3 colorscreen.

[lansing]

How did you limit FFMS2 to 1 thread ?


you do it in the core


core = vs.Core(threads=1)

[Myrsloik]

How did you limit FFMS2 to 1 thread ?
I've tried this :
core.std.LoadPlugin(path=r"k:\programs\ffms2-r725-icl\ffms2.dll", threads=1)
but I got the (atm not yet, but will soon be) famous 3 colorscreen.

Hint:
core.ffms2.Source(threads=1)

[Pat357]

for the dither one, you need to set mt=false on all manalyse, msuper, and mdegrain in order to work in vpy
It's not MVtools v2.6.05 that is the problem for vapoursynth, but the included "avstp.dll" is the one that causes problems.
Just rename this small "avptp.dll" file and you can use mvtools without problems. This way mvtools will fall back to single thread.
The "avstp.dll" makes that some filters ((re)written for this) capable of using multi-threading on their own (without using a MT version of Avisynth).
This makes it possible to have MT on standard Avisynth.

Info : avstp thread (http://forum.doom9.org/showthread.php?t=164407)

[Zarxrax]

Theoretical question here.

Over the years, several people tried and failed at creating a graphical Non Linear Editing system using AviSynth as a backend.
Might vapoursynth be better suited for this, and possibly hold some promise in that regard?

[Myrsloik]

Theoretical question here.

Over the years, several people tried and failed at creating a graphical Non Linear Editing system using AviSynth as a backend.
Might vapoursynth be better suited for this, and possibly hold some promise in that regard?

It was already quite possible with avisynth, the problem being that all those projects stayed as vaporware. Unless a skilled, sane person steps up (a rare happening indeed) I predict the same outcome as for avisynth.

So yes, very possible.

(Fun fact the, the toilet paper logo of ffms2 is actually a shameless insult of celluloid (http://forum.doom9.org/showthread.php?t=127187). Celluloid/cellulose, what's the difference?)

[lansing]

(Fun fact the, the toilet paper logo of ffms2 is actually a shameless insult of celluloid (http://forum.doom9.org/showthread.php?t=127187). Celluloid/cellulose, what's the difference?)

thanks for the reading, now i see where the word "vapor" is coming from. :)

http://forum.doom9.org/showthread.php?t=127187&page=3


And can you explain a bit more on how frame leveling multithreading works and how it compares to the current avisynth mt by SEt?

[Chikuzen]

I wrote an avisynth plugin to preview P216/P210/P016/P010 .vpy.

http://forum.doom9.org/showthread.php?t=166038

[kolak]

There is also real v210 preview from vpy via fake AVI (aka avfs) coming :)

update: v210 is shaping very well :)

[Myrsloik]

R12 is here. Changelog in the first page as usual.

The usual bloggy post here (http://www.vapoursynth.com/2012/10/r12-vapoursynth-takes-a-step-in-the-enterprise-direction/).

It would be nice if you redid all benchmarks with this version. It should be a bit faster than the previous ones where it couldn't reach 100% CPU usage.

[kolak]

Fake v210 AVI works great- most (if not all apps) read it fine. Fake AVI can be also loaded to Resolve through ref MOV file (way of passing AVC-I to Resolve if ffms2 can decode it).

[lansing]

benchmarks for r12.

mdegrain2 and mdegrain3 all have significant speed up comparing to r11, cpu usage reaching close to 99%. Mdegrain1 stayed the same.

[Myrsloik]

benchmarks for r12.

mdegrain2 and mdegrain3 all have significant speed up comparing to r11, cpu usage reaching close to 99%. Mdegrain1 stayed the same.

That's because mdegrain1 only has 3 filters and one source in total. Since only one frame can be processed by one filter at a time you become bottlenecked by the slowest filter.
If mdegrain1 speed is what you want at any cost then some tricks with splitting even and odd frame processing to different filter instances can be done with a clever script.

[Pat357]

MPC-HC with Madvr seems to crash when I drop this .vpy on MPC-window :

import vapoursynth as vs
import sys
core = vs.Core(accept_lowercase=True)
core.std.LoadPlugin(r'k:\programs\ffms2-r725-icl\ffms2.dll')
ret = core.ffms2.Source(r"k:\film\10bit\Madoka_v210.mkv")
last = ret
enable_v210=True


The error :

Faulting application name: mpc-hc.exe, version: 1.6.5.6055, time stamp: 0x506a1744
Faulting module name: vsvfw.dll, version: 0.0.0.0, time stamp: 0x506df320
Exception code: 0xc0000005
Fault offset: 0x00002fe0
Faulting process id: 0x10b8
Faulting application start time: 0x01cda31ff99abcae
Faulting application path: K:\programs\MPC-Homecinema-Lite\mpc-hc.exe


If I remove the last line in . vpy(= without "enable_v210=True"), the v210 file plays OK and shows up as P210 in Madvr.
This makes me think that the "v210 format" is the culprit somehow...
Is the VSVFW module not supposed to support v210 input ?
Any idea why this crash happens ?

If I create a fake AVI using PFM/VSFS.DLL, the resulting AVI plays OK and MadVR gives "v210" as format.
(with the line "enable_v210=True" included)

[Myrsloik]

If I remove the last line in . vpy, the v210 file plays OK and shows up as P210 in Madvr.
This makes me think that the "v210 format" is the culprit somehow...
Is the VSVFW module not supposed to support v210 input ?
Any idea why this crash happens ?

What's the resolution of the file?

test_stuff.zip (https://dl.dropbox.com/u/73468194/test_stuff.zip) <- you can also try these two replacement files and see if they help

[Pat357]

What's the resolution of the file?
It's full HD : 1920x1080
test_stuff.zip (https://dl.dropbox.com/u/73468194/test_stuff.zip) <- you can also try these two replacement files and see if they help
Thanks, will test it right away.

Tested the VSFS and VSVFW you've provided : All combinations old-new lead to crash when dropping .vpy on MPC.
As far as I can see, VSFS is OK, it seems VSVFW has problems. (of course, there might be interactions I don't know about)
If I remove the line "enable_v210=True", everything is OK.

Can you reproduce this problem ?

[kolak]

I've noticed the same- graphedit also crashes when vpy with v210 enabled is loaded, but when mounted than it works fine.
In the same time I'm yet to manage to crash any app with fake v210 AVI- this looks stable.

[mastrboy]

Is there any vapoursynth equilant command for histogram(mode="luma") ?
If not, is it planned?

[Myrsloik]

Is there any vapoursynth equilant command for histogram(mode="luma") ?
If not, is it planned?

Dig a few pages back for the same question and answer.

Also, I don't know what's planned. There is enough useful code in plugins for me to spend a whole year porting/improving it to vs standards.

This is where I suggest you "adopt" your favorite filter from avisynth and start coding.

[Myrsloik]

I turned a lot of the internal avisynth functions into a plugin. Here's a link (http://vapoursynth.com/downloads/avisynthfilters.7z) to it.

Note that Bob() (and possibly some other filters) won't work unless you also use the experimental vapoursynth.dll included that fixes several avisynth compat bugs.

I also had to add a tff=true/false argument to Bob() since vs doesn't keep track of that in the same way. This was just a quick delete key exercise so there may be more than one filter that doesn't work right.
Simply do print(core.avs.list_functions()) to see all the included fun.

The experimental vapoursynth.dll is in the archive.

[Pat357]

I turned a lot of the internal avisynth functions into a plugin. Here's a link (http://vapoursynth.com/downloads/avisynthfilters.7z) to it.

Simply do print(core.avs.list_functions()) to see all the included fun.
The experimental vapoursynth.dll is in the archive.
Thanks a lot !
I will create some scripts ; will post as soon as I encounter a problem.
Keep up building this marvelous tool !

Edit :
The docs mention YCOCG as "Color Family Constants", but it seems not to have a format. (can't specify 8bit / 420 sub-sampling).
How can I convert from RGB to YCOCG ? (I suppose this YCOCG is identical to YCgCo with U en V swapped ?)
The reason for my interest in this is the lossless conversion between YCgCo / RGB

[lansing]

I turned a lot of the internal avisynth functions into a plugin. Here's a link (http://vapoursynth.com/downloads/avisynthfilters.7z) to it.


Tested out. All functions work except bob, temporalsoften and spatialsoften, all returns "Avisynth Compat: requested frame xyz not prefetched, using slow method that may deadlock".

And there's no separate functions for assumettf/assumebff to be call

[Myrsloik]

Tested out. All functions work except bob, temporalsoften and spatialsoften, all returns "Avisynth Compat: requested frame xyz not prefetched, using slow method that may deadlock".

And there's no separate functions for assumettf/assumebff to be call

Of course there isn't. I even wrote that. I added the tff=true/false stuff to Bob just to get around it. If there is any other field order dependent function in there that I missed please tell me so I can hack it too.

I'll add those 3 functions to the internal prefect list so you won't get the prefetch warning. And remember, it's just a warning. If it doesn't get stuck completely the final result will always be fine.

[Reel.Deel]

@ Myrsloik
Thanks for the standalone Avisynth filters. I respect your decision on using the 2.5 interface. I think it would of been great if the the extra colorspaces in 2.6 would of been supported also.

On another note someone is currently porting histogram (https://github.com/dubhater/vapoursynth-histogram) and scxvid (https://github.com/dubhater/vapoursynth-scxvid) (never heard of it till today) to native Vapoursynth plugins. I wish I knew how to compile. :(

[Myrsloik]

@ Myrsloik
Thanks for the standalone Avisynth filters. I respect your decision on using the 2.5 interface. I think it would of been great if the the extra colorspaces in 2.6 would of been supported also.

On another note someone is currently porting histogram (https://github.com/dubhater/vapoursynth-histogram) and scxvid (https://github.com/dubhater/vapoursynth-scxvid) (never heard of it till today) to native Vapoursynth plugins. I wish I knew how to compile. :(

Avisynth 2.6 isn't even compatible with itself. I don't like moving targets. If some wants to add 2.6 support they're free to look at the 2.5 compat code and modify it. I'm not against it but I won't do it as I see it as a waste of time.

Maybe histogram works a little bit right now but scxvid definitely doesn't. We're not keeping secret stashes of plugins from you, they're just not ready.

[pbristow]

Myrsloik... :eek:

I'm almost speechless. I stopped paying much attention to the forum for a month or so (too much Real Life(TM) going on...), and I come back to find a revolution taking place! :)

This is an astonishing achievement for such a short space of time, a real lateral-thinking approach to circumventing the problems that have dogged Avisynth development in recent years that doesn't just look promising... it's already delivering!

(Haven't got it working on my own machine yet, mind you, but I will, I will... :o )

I look forward to future developments with great interest.

Bravo! :)

[Fullmetal Encoder]

benchmarks for r12.

mdegrain2 and mdegrain3 all have significant speed up comparing to r11, cpu usage reaching close to 99%. Mdegrain1 stayed the same.

Just to be clear, this is using the original filters? Assuming this, can we then expect a much greater advantage for VapourSynth over AviSynth with those filters specifically re-written to take advantage of VapourSynth's multithreading scheme?

[Myrsloik]

Just to be clear, this is using the original filters? Assuming this, can we then expect a much greater advantage for VapourSynth over AviSynth with those filters specifically re-written to take advantage of VapourSynth's multithreading scheme?

No, this is it. You won't gain any speed at all from porting them. You'll simply have to optimize the code more. Sad truth.
And yes, those benchmarks were made with the original avisynth binaries.

[SassBot]

In case anyone is interested I have a git mirror of the vapoursynth code on Github here (https://github.com/amichaelt/vapoursynth). Thanks for the awesome work, too, Myrsloik. At some point if you or someone else doesn't tackle it I was going to look into see what would be needed to support 2.6 Avisynth plugins.

[lansing]

Just to be clear, this is using the original filters? Assuming this, can we then expect a much greater advantage for VapourSynth over AviSynth with those filters specifically re-written to take advantage of VapourSynth's multithreading scheme?

well you cannot expect any greater speed when your cpu was already running at 99%. If you're looking for more speed, get a better cpu. I'm pretty sure an intel i7 can easily blown away my amd 4 core.

Memory wise, vapoursynth will beat avisynth in any instance. I can recall the many times that i got pissed when trying to encode my bluray video, and avisynth crashed on me because of the "out of memory" thing, and i have 12G RAM on my machine.

[mandarinka]

Avisynth obviously can't use more than 4 GB. It's 32-bit a program.

[Groucho2004]

Avisynth obviously can't use more than 4 GB. It's 32-bit a program.

The address space for a 32 bit process on 32 bit Windows is 2 GB. With some hacking this can be extended to 4GB, but only on 64 bit Windows.

[LoRd_MuldeR]

It doesn't require "hacking", it just needs adding the /LARGEADDRESSAWARE option when compiling the executable.

And of course the code must able to deal with addresses that have the most significant bit set. Just setting the /LARGEADDRESSAWARE flag for arbitrary code might break it!

Because Avisynth is a DLL that gets loaded into the process of some other application, whether /LARGEADDRESSAWARE is set or not totally depends on how that application was compiled.

BTW: On 32-Bit systems you can still get 3 GB virtual address space with /LARGEADDRESSAWARE, but it requires a special Windows boot option.

The full 4 GB are indeed only possible for 32-Bit processes when compiled with /LARGEADDRESSAWARE running and when running on a 64-Bit operating system...

[CarlPig]

Im very newbie to Vapoursynth, so i hope someone can help me with a step by step guide.

1. How to make this avs script to vs script (I have a 2 core cpu with 2 threads):
LoadPlugin("C:\Program Files (x86)\MeGUI\tools\dgindexnv\DGDecodeNV.dll")
DGSource("D:\Bluray Dumps\Fast.Og.Furious.2009\Fast.Og.Furious.2009.dgi",fieldop=0, deinterlace=1)
crop(2, 140, -2, -148)

2. How to open a vs script in VirtualDub

3. How to make this avs script to vs script:
AviSource("D:\Bluray Dumps\Fast.Og.Furious.2009.vdr")
converttoYV12()
hqdn3d(4.0,3.0,6.0,4.5).f3kdb
hqdn3d(4.0,4.0,1.0,1.0).f3kdb
LSFMod(dest_x=1920, dest_y=824)

4. How to send the vs script to x264

[LoRd_MuldeR]

CarlPig, have you already looked at the documentation?
ttp://www.vapoursynth.com/doc/index.html

And at this post, which explains usage with x264?
http://forum.doom9.org/showpost.php?p=1521791&postcount=1

:readguid:

[Groucho2004]

It doesn't require "hacking", it just needs adding the /LARGEADDRESSAWARE option when compiling the executable
Semantics... :)

BTW: On 32-Bit systems you can still get 3 GB virtual address space with /LARGEADDRESSAWARE, but it requires a special Windows boot option.
I choose to ignore that option, never got it to work properly and from what I've read I'm not the only one.

[Chikuzen]

I wrote rawsource plugin for VapourSynth.
vsrawsource-0.1.0.7z (http://www.mediafire.com/download.php?xfr11bbuv0qi3ze)

BTW, I think that each plugin should have its own thread.
However, I'm not clear about where is the right place for VS plugins.
Since vsavsreader had relation very strong against avisynth, there was no problem in particular at the present place.

[Myrsloik]

I wrote rawsource plugin for VapourSynth.
vsrawsource-0.1.0.7z (http://www.mediafire.com/download.php?xfr11bbuv0qi3ze)

BTW, I think that each plugin should have it's own thread.
However, I'm not clear about where is the right place for VS plugins.
Since vsavsreader had relation very strong against avisynth, there was no problem in particular at the present place.

Create all the threads you want, the worst thing that could happen is that a moderator splits or renames the forum name to avisynth and vs I guess.

Once the raw source plugin has been tested and is stable I think I'd like to include it into the vs source tree if you don't mind and I'll add a link to it in the vs documentation in the next release.

[Chikuzen]

Create all the threads you want, the worst thing that could happen is that a moderator splits or renames the forum name to avisynth and vs I guess.

ok, done (http://forum.doom9.org/showthread.php?t=166075).

Once the raw source plugin has been tested and is stable I think I'd like to include it into the vs source tree if you don't mind and I'll add a link to it in the vs documentation in the next release.
no problem

[SassBot]

Hey Myrsloik have you written or do you plan on writing any sort of documentation on porting over Avisynth plugins to Vapoursynth? I understand you are busy and I'm definitely going to dig into the code but at least some basic guidelines for porting over would be helpful.

[SassBot]

So I'm taking a crack at porting ColorMatrix to Vapoursynth and I have a question about how to handle the fact that it requests a frame within the constructor. I see that the vsapi->getFrame says:

do never use inside a filter's getframe function, for external applications using the core as a library or for requesting frames in a filter constructor

So do I use the vsapi->newVideoFrame instead or how should I handle this?

To add the code section I'm dealing with is this:


if (hints)
{
int temp;
child->SetCacheHints(CACHE_RANGE, 1);
hintClip = child;
PVideoFrame pv = child->GetFrame(0, env);
getHint(pv->GetReadPtr(PLANAR_Y), temp);
if (temp == -1)
env->ThrowError("ColorMatrix: no hints detected in stream with hints=true!");
}

[Myrsloik]

Hey Myrsloik have you written or do you plan on writing any sort of documentation on porting over Avisynth plugins to Vapoursynth? I understand you are busy and I'm definitely going to dig into the code but at least some basic guidelines for porting over would be helpful.

You are allowed to use getframe in the constructor if you really have to. But in colormatrix you probably shouldn't. Just wait until the first frame request and initialize it then. Really, you'll have to initialized it any time the colorimetry changes anyway.

You can find a well commented invert filter example in trunk/sdk in the source tree.

[SassBot]

Yeah after I posted and reading more of the ColorMatrix code I noticed that it was basically doing the same check in its getframe function as well anyway. No real point in doing it twice.

[SassBot]

Since I've been unable to find any examples of its use within your example plugins or any of the ported plugins: What is the proper way to call internal functions from within a Vapoursynth plugin? I see there is an invoke function on the VSAPI object but where do I get the VSPlugin pointer from? Is that the object that you get passed in from the VapourSynthPluginInit function?

[Myrsloik]

Since I've been unable to find any examples of its use within your example plugins or any of the ported plugins: What is the proper way to call internal functions from within a Vapoursynth plugin? I see there is an invoke function on the VSAPI object but where do I get the VSPlugin pointer from? Is that the object that you get passed in from the VapourSynthPluginInit function?

I don't think I use that in any simple code. Basically if you want to use a built-in function you use getPluginId to obtain the VSPlugin pointer to the plugin you want. Use com.vapoursynth.std as the identifier if you want to call an internal function.

Now I'm curious, why do you need this in colormatrix?

[SassBot]

Trying to figure out the best way to port over the invoke calls it makes for SeparateFields and Weave for interlaced footage and to replace the Limiter call to a call to Lut. I figured if there are already internal functions for them no need to duplicate.

[Myrsloik]

Trying to figure out the best way to port over the invoke calls it makes for SeparateFields and Weave for interlaced footage and to replace the Limiter call to a call to Lut.
I can accept the separatefields and weave stuff. Seems reasonable.
But I will slap you with a Norwegian salmon if you add on a lut at the end for range limiting.
What you should do is add a pmaxub/pminub at the beginning and end of the already existing pile of inline asm. Using a lut for what is in practice only 2 cpu instructions is absurd.

[SassBot]

Hehe. Well the limiter switch was just throwing out based on what you said to replace the call with in the docs. But, that is a better idea for sure. :-)

[TheFluff]

While you're at it, replace the hinting shit with custom per-frame properties.

[SassBot]

I'll look into that once I get it up and running.

[CarlPig]

CarlPig, have you already looked at the documentation?
ttp://www.vapoursynth.com/doc/index.html

And at this post, which explains usage with x264?
http://forum.doom9.org/showpost.php?p=1521791&postcount=1

:readguid:

I have now looked at the documentation.

But can't still not figure out of that, so i hope someone will help me start.

I hope its spelled correctly, because im not so good to english.

[SassBot]

Can't figure what out? The first post made by Myrsloik tells you how to use it with x264:


The easiest way to pipe to x264:
1. Import the sys module in your python script.
2. To output a clip to stdout simply add "someclip.output(sys.stdout, y4m=True)" to your script.
3. Run the script with this command: python script.py | x264 --demuxer y4m --output encodedfile.mkv -
(only works for yuv formats with an y4m header value)

There are also numerous script examples throughout this thread in addition to this sample in the documentation here (http://www.vapoursynth.com/doc/gettingstarted.html). How much more explicit does it need to be?

[mandarinka]

So I'm taking a crack at porting ColorMatrix to Vapoursynth

Color adjustment like this is a kind of operation that would benefit from being able to work in/output higher than 8-bit precision data. Wouldn't it be better to build a solution around Dither tools?

Currently you can script colormatrix conversion with it, but it forces you to upsample and downsample chroma.

[SassBot]

Since ColorMatrix is used almost exclusively for sources that are 8-bit and will be reencoded as 8-bit what would be the point? Also, it's more of a way for me to refresh my knowledge on Avisynth plugin development and get into Vapoursynth plugin development. If you want to make something based on dither tools for Vapoursynth to do the same thing, go ahead. My interest is in the learning.

[Myrsloik]

It's time for another release! This time it's the equivalent of conditional filtering that gets a huge improvement and I enabled more of the cache memory management now that I have more complex scripts to test with. I also added a slightly revised version of the avisynth core filters to the downloads.

As usual the bloggy post is here (http://www.vapoursynth.com/2012/10/r13-conditional-filtering-and-memory-optimizations/) and consider adopting your favorite avisynth filter and porting it.

You should also take a look at Chikuzen's filters if you haven't done so already.

[Are_]

I don't know if I'm doing something wrong, but r13 module does not load in Python.

I get the following error on wine:

$ ./vs-enc.sh test.vpy
err:module:import_dll Library MSVCR100D.dll (which is needed by L"C:\\Python33\\lib\\site-packages\\vapoursynth.dll") not found
err:module:import_dll Library vapoursynth.dll (which is needed by L"C:\\Python33\\lib\\site-packages\\vapoursynth.pyd") not found
Traceback (most recent call last):
File "test.vpy", line 4, in <module>
import vapoursynth as vs
ImportError: DLL load failed: Module not found.
x264 [error]: could not open input file `-'

I tried too on Windows 7 and fails to load too, but less verbose.
And vcredist 2010 is installed in both machines. R12 loads fine.

[Myrsloik]

Once again I mess up use a debug compile. The r13 installer has been updated.

Also, why are you using vs in wine?

[Are_]

Thx Myrsloik.

Because I use Linux as my desktop mainly, personal preferences I suppose, and if I compile it natively I get stuck with native filters (64bit, although this is a guess on my side).
Anyway, don't worry, if something fails there I make sure it's not OS related before reporting.

[SassBot]

So outside of recompiling Vapoursynth is there any way I can test my ported ColorMatrix since you have it blacklisted? :p

[Myrsloik]

So outside of recompiling Vapoursynth is there any way I can test my ported ColorMatrix since you have it blacklisted? :p

Blacklisted? I only print a warning when you load the original avisynth one. I crashes so well on its own I never saw a reason to actively stop it.

[SassBot]

Ok, nm. It was an unrelated error throwing up.

[kolak]

import vapoursynth as vs
import sys
core = vs.Core(threads=6)
core.std.LoadPlugin(path=r'C:\Program Files (x86)\VapourSynth\ffms2.dll')
ret = core.ffms2.Source(source=r'S:\temp\Avatar_ProRes_L.mov',threads=6)
a= core.resize.Spline(clip=ret, width=720, height=576,format=vs.YUV422P10)
last= core.resize.Spline(clip=a, width=1920, height=1080,format=vs.YUV422P10)
enable_v210=True

compared to:

import vapoursynth as vs
import sys
core = vs.Core(threads=6)
core.std.LoadPlugin(path=r'C:\Program Files (x86)\VapourSynth\ffms2.dll')
ret = core.ffms2.Source(source=r'S:\temp\Avatar_ProRes_L.mov',threads=6)
a= core.resize.Spline(clip=ret, width=720, height=576,format=vs.YUV422P10)
last= core.resize.Spline(clip=a, width=1920, height=1080,format=vs.COMPATYUY2)
#enable_v210=True

runs about 4x faster (1st one is faster). Why?

[Myrsloik]

import vapoursynth as vs
import sys
core = vs.Core(threads=6)
core.std.LoadPlugin(path=r'C:\Program Files (x86)\VapourSynth\ffms2.dll')
ret = core.ffms2.Source(source=r'S:\temp\Avatar_ProRes_L.mov',threads=6)
a= core.resize.Spline(clip=ret, width=720, height=576,format=vs.YUV422P10)
last= core.resize.Spline(clip=a, width=1920, height=1080,format=vs.YUV422P10)
enable_v210=True

compared to:

import vapoursynth as vs
import sys
core = vs.Core(threads=6)
core.std.LoadPlugin(path=r'C:\Program Files (x86)\VapourSynth\ffms2.dll')
ret = core.ffms2.Source(source=r'S:\temp\Avatar_ProRes_L.mov',threads=6)
a= core.resize.Spline(clip=ret, width=720, height=576,format=vs.YUV422P10)
last= core.resize.Spline(clip=a, width=1920, height=1080,format=vs.COMPATYUY2)
#enable_v210=True

runs about 4x faster (1st one is faster). Why?

It's swscale. Its design makes no sense at all. I can however tell you that some conversion paths are "optimized". Nobody will give you a clearer answer than this since the code is so unreadable.

[SassBot]

Quick question, ColorMatrix does a check when processing interlaced footage to see if it's FieldBased and it errors out if it is. Is this check even still relevant or necessary for VapourSynth?

[Myrsloik]

Quick question, ColorMatrix does a check when processing interlaced footage to see if it's FieldBased and it errors out if it is. Is this check even still relevant or necessary for VapourSynth?

I have a different idea about field handling. The user knows best. So let the users mess up if they want to.

Maybe I'll specify some frame property for specifying frame/field based but I don't think it's necessary judging by how random it is in avisynth anyway.

[SassBot]

Okay, then I'll just omit the check. It can be fixed if leaving it out actually causes an issue for someone.

[Pat357]

a= core.resize.Spline(clip=ret, width=720, height=576,format=vs.YUV422P10)
last= core.resize.Spline(clip=a, width=1920, height=1080,format=vs.YUV422P10)
enable_v210=True

compared to:

a= core.resize.Spline(clip=ret, width=720, height=576,format=vs.YUV422P10)
last= core.resize.Spline(clip=a, width=1920, height=1080,format=vs.COMPATYUY2)
#enable_v210=True

runs about 4x faster (1st one is faster). Why?


It depends on the source format; In the first, you do :
Source -> YUV422P10 -> YUV422P10

In the second you do an additional conversion :
Source -> YUV422P10 -> COMPATYUY2
This is a conversion from a 10b format (YUV422P10) to 8 bit packed format. I'm not sure if swscale does it, but this should also be dithered of course.

This is 0 conversions for first case against 1 conversion for 2e case. (assuming source is also YUV422P10)

Why do you scale in 2 "steps" ?

[mandarinka]

Since ColorMatrix is used almost exclusively for sources that are 8-bit and will be reencoded as 8-bit what would be the point?

To prevent banding. Obviously you can encode to 10bit afterwards, too!

In my usage, sources that needed colormatrix conversion needed level conversion too*, and with two such adjustments, the odds of banding really increase.

* They were DVD sources that in fact had pc levels and bt709 colormatrix. True, you can solve this by flagging the h.264 streams, but then almost no playback environment respects those. Hence why you sorta need to actually convert the video.

[SassBot]

How exactly does ColorMatrix introduce banding or how would using anything greater than 8-bit within ColorMatrix prevent it? Why would using greater than 8-bit for a filter that is only really for 8-bit sources going to do anything useful?

I can see adding support for other colorspaces, but I still don't see how processing say YV12 at 10-bit or higher is going to do anything useful as I've never seen nor heard anyone complain that ColorMatrix introduces banding.

[Myrsloik]

How exactly does ColorMatrix introduce banding or how would using anything greater than 8-bit within ColorMatrix prevent it? Why would using greater than 8-bit for a filter that is only really for 8-bit sources going to do anything useful?

I can see adding support for other colorspaces, but how would processing YV12 at greater than 8-bit do anything useful?

It's most likely simple rounding errors. You can see them surprisingly quickly if you only have 8 bits per channel.

I hope the world will switch to something like 16bit precision when processing things soon...

[SassBot]

I guess, but unless someone is seeing drastic issues with ColorMatrix adding banding I don't think implementing the processing at 10-bit or higher for YV12 is really useful effort. If it is it would be the first time I've ever heard of it doing so.

Edit to add: Yes, I realize the theoretical issue that can crop up by rounding errors, but unless someone can show some actual banding issues I'm not going to spend my time writing a solution for a theorectical problem that I've never seen with my eyes. But at this point I'm not going to derail the thread on that topic anymore.

[kolak]

This is a conversion from a 10b format (YUV422P10) to 8 bit packed format. I'm not sure if swscale does it, but this should also be dithered of course.

Why do you scale in 2 "steps" ?

There is no specific need in my scripts- it was done just for testing.

It's bit strange that scaling + 10 to 8bit conversion is so slow.

When you do one scaling:

a= core.resize.Spline(clip=ret, width=720, height=576,format=vs.YUV422P10)

and than:

a= core.resize.Spline(clip=ret, width=720, height=576,format=vs.COMPATYUY2)

than speed is about the same (even if ProRes source gets decoded to 10bit). As Myrsloik said- there is some mess in swscale.

[active1]

i have a problem with vapoursynth on linux, it crashes after encoding 1 frame (when piping to x264-10bit)

python3 test.py | x264 --demuxer y4m --output encodedfile.mkv -
y4m [info]: 1280x720p 0:0 @ 3908/143 fps (cfr)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 FastShuffle SSE4.2
x264 [info]: profile High 10, level 3.1, 4:2:0 10-bit
y4m [error]: bad header magic (400040 <=> @)
y4m [error]: bad header magic (40004000 <=> )
x264 [info]: frame I:1 Avg QP:45.00 size: 2350
x264 [info]: mb I I16..4: 97.6% 0.0% 2.4%
x264 [info]: 8x8 transform intra:0.0%
x264 [info]: coded y,uvDC,uvAC intra: 1.4% 2.2% 2.2%
x264 [info]: i16 v,h,dc,p: 100% 0% 0% 0%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 76% 0% 24% 0% 0% 0% 0% 0% 0%
x264 [info]: i8c dc,h,v,p: 2% 0% 98% 0%
x264 [info]: kb/s:513.78

encoded 1 frames, 8.47 fps, 668.13 kb/s
Traceback (most recent call last):
File "test.py", line 6, in <module>
v.output(sys.stdout, y4m=True)
File "vapoursynth.pyx", line 658, in vapoursynth.VideoNode.output (build/src/cython/vapoursynth.c:10442)
vapoursynth.Error: 'write() call returned error'



script:
import vapoursynth as vs
import sys
c = vs.Core()
c.std.LoadPlugin('/usr/local/lib/libffms2.so')
v=c.ffms2.Source('/home/200.mkv')
v.output(sys.stdout, y4m=True)


and this is the clip (https://www.dropbox.com/s/3m3pr61ycoq33my/200.mkv)
there is nothing special in that clip, and that is the problem :(
i always have this problem, but sometimes more than 1 frame encoded
and i tried to view script by piping it to avplay, then all what i get is a green window

[JEEB]

...
Not in any way Linux specific, and these extensions aren't yet in libav as far as I know.

I wrote the support for the high bit depth Y4M extensions into x264's y4m input module only some time ago, and that stuff isn't in the main repository yet, so trying to pass it as y4m will fail right now unless you apply the patch (https://github.com/DarkShikari/x264-devel/commit/a7530d18bef4a32c79e60094e4f6195ef333057f.patch).

[qyot27]

Since the extensions are on FFmpeg's side, it is possible to pipe from python, through ffmpeg, and then to x264. Of course, that last pipe wouldn't be necessary if ffmpeg was built with libx264 linked in. Might also be the only way to preview it using mplayer, as I can't remember if one linked against ffmpeg inherits the support for higher bit depths too, or if that's strictly internal. Although at that point you could also use ffplay to do it.
python3 testscript.vpy | ffmpeg -i - -f yuv4mpegpipe - | x264 [options as usual]
or
python3 testscript.vpy | ffmpeg -i - -vcodec libx264 -preset ultrafast -tune zerolatency -crf 18 outputfile

EDIT: Yeah, that totally doesn't work when dealing with high bit depths. Took me this long to get around to test it. ffmpeg can turn it into a compatible rawvideo stream, though. So it doesn't change the fact that getting y4m+hbd to work right with x264 will have to wait on that patch.

[Chikuzen]

Why everyone want to use shell ?
I think that one of the big advantage of VapourSynth is that it can control by Python (https://gist.github.com/3804501) easily :rolleyes:

[Pat357]

Since the extensions are on FFmpeg's side, it is possible to pipe from python, through ffmpeg, and then to x264.
python3 testscript.vpy | ffmpeg -i - -f yuv4mpegpipe - | x264 [options as usual]

How would this extra piping through ffmpeg change the 10 or 16bit format ? FFmpeg gets Y4M , does it output also Y4M via the pipe to x264 ?
What format does x264 get ? Still y4m ?

[Chikuzen]

After update VS to r13, I got an issue.


#sample.vpy
import vapoursynth as vs
core = vs.Core()
core.std.LoadPlugin('G:/vsplugins/vsrawsource.dll')
clip = core.raws.Source('D:/test_media/soccer.y4m')
last = clip


#sample.avs
AVISource('sample.vpy')


> x264.exe ./sample.avs -o out.h264


This works normally on r12.
However, on r13, x264 (and the other tools using avisynth_c interface like avs2yuv, avs2pipemod, etc) stops after displaying the message "Bad fpu state detected after plugin load"

what is "Bad fpu state" ?
How should I do ?

[Myrsloik]

After update VS to r13, I got an issue.


#sample.vpy
import vapoursynth as vs
core = vs.Core()
core.std.LoadPlugin('G:/vsplugins/vsrawsource.dll')
clip = core.raws.Source('D:/test_media/soccer.y4m')
last = clip


#sample.avs
AVISource('sample.vpy')


> x264.exe ./sample.avs -o out.h264


This works normally on r12.
However, on r13, x264 (and the other tools using avisynth_c interface like avs2yuv, avs2pipemod, etc) stops after displaying the message "Bad fpu state detected after plugin load"

what is "Bad fpu state" ?
How should I do ?
Which compiler did those other tools use? If I'm right then recompiling them with msvc will fix it.
You get that error when when something messes up the fpu control word. Some compilers change it just for fun. See this post for a longer explanation: http://virtualdub.org/blog/pivot/entry.php?id=53
I suppose my strict check is too much for the avisynth mess...

[Chikuzen]

Which compiler did those other tools use?
maybe, mingw-w64-gcc.
I downloaded it from x264.nl (compiled by JEEB).
If I'm right then recompiling them with msvc will fix it.
How can I compile x264 with MSVC ?
It's not libav.

[Myrsloik]

maybe, mingw-w64-gcc.
I downloaded it from x264.nl (compiled by JEEB).

How can I compile x264 with MSVC ?
It's not libav.

Hmm... I'll think of something for the next version... I'll probably just replace that check with a huge warning in R14.

[Myrsloik]

Here's my latest creation: VIVTC (https://dl.dropbox.com/u/73468194/vivtc.dll)
The matching metric calculation is directly based on TFM but all the logic around has been rewritten, parts of it estimated from the tfm manual's vague descriptions. VDecimate is written from scratch and uses a similar method to both decomb and tdecimate.

See it as a portable "TIVTC lite" with most features cut since nobody changes the defaults anyway. The only real new feature is that VFM has support for directly applying its matches to another clip (use the clip2 argument for that).

I also wrote a bloggy post (http://www.vapoursynth.com/2012/10/open-binary-introducing-a-practical-alternative-to-open-source/) that tries to be funny. It's about open source in its more degenerate forms.

[wOxxOm]

So VFM uses clip2 differently from TFM which was using it for combed frames only? Also, 'ovr', 'output' parameters were immensely useful. And I wonder if is it possible to display a tfm(display=true)-like video overlay with frame stats like MI, field match, etc

[Myrsloik]

So VFM uses clip2 differently from TFM which was using it for combed frames only? Also, 'ovr', 'output' parameters were immensely useful. And I wonder if is it possible to display a tfm(display=true)-like video overlay with frame stats like MI, field match, etc

Correct. It has no built in pp and clip2 is used to apply matches to. I will make it hint pp stuff properly later. It already attaches all calculated metrics to the output frames though. Maybe I'll reimplement some additional features later that I feel are missing, like ovr. Actually that's the only one I feel is missing. Maybe display once some other things are into place... other people are working on some interesting things for vs.

If you want tfm then use tfm. I'm not going to add that many features. I suggest you always use vdecimate in vs though since it should be faster.

Oh,I believe vivtc in its current state is good enough for 95% of the tivtc users. Patches are of course accepted if you want to add a specific feature yourself. See my latest bloggy post for examples on why I junked tivtc and started over instead of porting it.

[wOxxOm]

Thank you. I liked that blog post :)
Frame properties would be quite enough I suppose _if_ those could be used in a user defined function, like clip2=userfunc(statsclip) for example or somehow else, to selectively deint/postprocess some frames based on metrics. As for displaying metrics: if those could be read as frame properties then it'll be possible to make a user function that uses Subtitle to output them I guess, but not that rectangle that tfm draws where it finds combing. Anyway, I agree, your current implementation covers almost 100% of commonly used functionality.

p.s. It occured to me that there's probably no need to pass stats to clip2 user function if I can iterate through frames and apply processing depending on frame properties added previously by VFM.

[sneaker_ger]

Since the number of frames has been reduced it probably only forgot to AssumeFPS afterwards?

[Myrsloik]

Since the number of frames has been reduced it probably only forgot to AssumeFPS afterwards?

Right, I forgot the fps... there are also some other small details left to fix. My next project will be something that adds a bit more masktools/overlay functionality I think

[Chikuzen]

@Myrsloik
How about adding new color family which has four planes ?
I think that it is useful for alpha support.

[Myrsloik]

@Myrsloik
How about adding new color family which has four planes ?
I think that it is useful for alpha support.

I'm thinking about it. It probably should be added in some way... but it's complicated. Sometimes alpha should be processed too (like trim, splice and such) but should be ignored for filtering operations to not be ruined. That's why I thought attaching it as a metadata frame or using mask clips was the best idea at first. Right now I have no good idea for a clean solution...
All I know is that I want to avoid avisynth's mess of alpha vs mask clip.

There's also a new experimental build of vdub available with .vpy support here (http://forums.virtualdub.org/index.php?act=ST&f=15&t=21216&).

[kolak]

For some reason new Vdub "feels" working faster - hehe

[cihub]

Myrsloik, can you change this string in a first post:
"python script.py | x264 --demuxer y4m --output encodedfile.mkv -"
to:
"python script.py | x264 --demuxer y4m --output encodedfile.mkv - "
For noobs like myself that missing space at the end of the command can lead to a great confusion...

P.S. to those who new to this piece of software and tried to do exactly like it was written here (http://www.vapoursynth.com/doc/gettingstarted.html)
if smth like "ret = core.ffms2.Source(source='Super Size Me.avi')" are not working for you (i'm using R13 with ffms2.dll from MeGUI), try this instead: "ret = core.avs.FFVideoSource(r'd:\path\to\video_file.mkv')".

[Myrsloik]

Myrsloik, can you change this string in a first post:
"python script.py | x264 --demuxer y4m --output encodedfile.mkv -"
to:
"python script.py | x264 --demuxer y4m --output encodedfile.mkv - "
For noobs like myself that missing space at the end of the command can lead to a great confusion...

P.S. to those who new to this piece of software and tried to do exactly like it was written here (http://www.vapoursynth.com/doc/gettingstarted.html)
if smth like "ret = core.ffms2.Source(source='Super Size Me.avi')" are not working for you (i'm using R13 with ffms2.dll from MeGUI), try this instead: "ret = core.avs.FFVideoSource(r'd:\path\to\video_file.mkv')". Oh, and don't forget to set resolution explicitly (in my case it used 466x704 instead of 960x720).

Since when does it need a space? I'll have to try this myself when I get home.
I also assume minimal python proficiency. As in knowing how to escape a string. There's a reason the first thing I suggest on the very same page is to check out a python tutorial.

Oh, and the project isn't dead. I just have a cold.

[qyot27]

With the official binary of R13 I'm consistently getting a crash after frame #99. It affects all the files I've tested it with (tested with .avi, .mp4, .mov, .mkv, and .flv files...still results in x264 only getting 99 frames to encode before crashing). The Windows error reporting dialog says the crash is in vapoursynth.dll, so is it safe to say that's where it really is? Namely because the script only uses FFMS2, so if it was actually in the source filter I'd think it would report that ffms2.dll crashed. I tried culling what little information gdb can generate without debug symbols, but that was completely fruitless (tried hooking both python itself and x264...neither one gave me anything to go on).

None of this happens on Linux, so it seems like it might just be a problem with compilation settings or something (I'll have to try satisfying the dependencies first to attempt building under Windows, though). Unless this is part of a fix that occurred between the release of R13 and HEAD.

Is there any kind of internal error reporting type of thing that could be used to help diagnose this?


Some output from the Command Prompt:
>python vpytestscript.vpy | x264 --stdin y4m --preset ultrafast --crf 18 -o testvs.mkv -
y4m [info]: 1920x800p 0:0 @ 24000/1001 fps (cfr)
y4m [info]: color matrix: undef
x264 [info]: using cpu capabilities: MMX2 Cache32
x264 [info]: profile Constrained Baseline, level 4.0
x264 [info]: started at Sat Oct 27 18:04:29 2012

x264 [info]: frame I:1 Avg QP:15.00 size: 92219
x264 [info]: frame P:98 Avg QP: 7.36 size: 891
x264 [info]: mb I I16..4: 100.0% 0.0% 0.0%
x264 [info]: mb P I16..4: 0.0% 0.0% 0.0% P16..4: 1.9% 0.0% 0.0% 0.0% 0.0% skip:98.1%
x264 [info]: coded y,uvDC,uvAC intra: 11.2% 10.7% 10.7% inter: 0.6% 0.4% 0.4%
x264 [info]: i16 v,h,dc,p: 92% 6% 2% 0%
x264 [info]: i8c dc,h,v,p: 89% 7% 3% 0%
x264 [info]: kb/s:347.87

encoded 99 frames, 2.540 fps, 349.09 kb/s, 175.96 KB
x264 [info]: ended at Sat Oct 27 18:05:08 2012
x264 [info]: encoding duration 0:00:39

And one with the normal AviSynth side of the .dll to prove there's nothing wrong there. I killed it at 151 frames so I wouldn't have to wait around too long.
>x264 --preset ultrafast --crf 18 -o testvs-avs.mkv testvs.avs
avs [info]: 1920x800p 0:0 @ 24000/1001 fps (cfr)
avs [info]: color matrix: undef
x264 [info]: using cpu capabilities: MMX2 Cache32
x264 [info]: profile Constrained Baseline, level 4.0
x264 [info]: started at Sat Oct 27 18:05:54 2012
[5.0%] 151/3008 frames, 5.969 fps, 249.76 kb/s, 192.02 KB, eta 0:07:58, est.size

x264 [info]: frame I:1 Avg QP:15.00 size: 92219
x264 [info]: frame P:150 Avg QP: 7.27 size: 692
x264 [info]: mb I I16..4: 100.0% 0.0% 0.0%
x264 [info]: mb P I16..4: 0.7% 0.0% 0.0% P16..4: 1.5% 0.0% 0.0% 0.0% 0.0% skip:97.9%
x264 [info]: coded y,uvDC,uvAC intra: 5.7% 5.4% 5.3% inter: 0.5% 0.3% 0.2%
x264 [info]: i16 v,h,dc,p: 95% 3% 2% 0%
x264 [info]: i8c dc,h,v,p: 95% 4% 2% 0%
x264 [info]: kb/s:248.96

aborted at input frame 151, output frame 151
encoded 151 frames, 5.969 fps, 249.76 kb/s, 192.02 KB
x264 [info]: ended at Sat Oct 27 18:06:20 2012
x264 [info]: encoding duration 0:00:26

>

[Myrsloik]

With the official binary of R13 I'm consistently getting a crash after frame #99. It affects all the files I've tested it with (tested with .avi, .mp4, .mov, .mkv, and .flv files...still results in x264 only getting 99 frames to encode before crashing). The Windows error reporting dialog says the crash is in vapoursynth.dll, so is it safe to say that's where it really is? Namely because the script only uses FFMS2, so if it was actually in the source filter I'd think it would report that ffms2.dll crashed. I tried culling what little information gdb can generate without debug symbols, but that was completely fruitless (tried hooking both python itself and x264...neither one gave me anything to go on).

None of this happens on Linux, so it seems like it might just be a problem with compilation settings or something (I'll have to try satisfying the dependencies first to attempt building under Windows, though). Unless this is part of a fix that occurred between the release of R13 and HEAD.

Is there any kind of internal error reporting type of thing that could be used to help diagnose this?



Vs always outputs an error message for the fatal errors it has checks for, which are quite a few. There's no additional debug output as I like to make warnings visible for everyone to see.
I'm just going to guess that it's either a bad ffms2 compile or you've found a new bug. If you point me at the exact script and ffms2 build I'll test it as soon as my cold is over. Sigh... I really want to finish up r14 soon...

[qyot27]

The FFMS2 build was the one I posted on the 12th (the post itself (http://forum.doom9.org/showthread.php?p=1595488#post1595488)).

The script:
import vapoursynth as vs
import sys
core = vs.Core()
core.std.LoadPlugin(r"C:\Program Files\AviSynth 2.5\plugins\ffms2.dll")
ret = core.ffms2.Source("ironman3-tlr1-m4mb0_h1080p.mov")
ret.output(sys.stdout, y4m=True)

[active1]

hi, i have a transport stream and i want to ivtc it and encode it as vfr.
i can do that easily in avisynth with TIVTC by outputting the timecodes and then encode it in x264, but how can i do that in vs with VIVTC?

[Myrsloik]

hi, i have a transport stream and i want to ivtc it and encode it as vfr.
i can do that easily in avisynth with TIVTC by outputting the timecodes and then encode it in x264, but how can i do that in vs with VIVTC?

You can't. I suggest you use TFM+VDecimate (I think that combination can give you vfr too but I could be mistaken in how TIVTC works). The auto vfr stuff I just thought was one of those niche features few people use.

Peronally I'm more used to people loading anything that's vfr in yatta and then marking stuff there. Where did you find a vfr transport stream anyway?

[active1]

You can't. I suggest you use TFM+VDecimate (I think that combination can give you vfr too but I could be mistaken in how TIVTC works).


i used the 2-pass vfr way in this (http://avisynth.org/mediawiki/VFR) page with TIVTC every time i had a hybrid transform stream because i don't want to blend any frames and i need "input, tfmin, and mkvout" parameters to do this with VDecimate

The auto vfr stuff I just thought was one of those niche features few people use.
but how it works?

Peronally I'm more used to people loading anything that's vfr in yatta and then marking stuff there. Where did you find a vfr transport stream anyway?
i don't think that i need to use yatta because it's easy to do what i need in avisynth only. its a hybrid transform stream

[Reel.Deel]

As some you guys probably already know, dubhater started porting a couple of plugins over to VS. You can now download a windows dll for both Histogram (https://github.com/dubhater/vapoursynth-histogram/downloads) and TemporalSoften (https://github.com/dubhater/vapoursynth-temporalsoften/downloads).

*edit*
Right after my post, dubhater updated Histogram. Classic mode works with up to 16 bits now.

Also, 06_taro posted (http://astrataro.wordpress.com/2012/10/27/x264-hi-bpp-y4m/) a special build of x264. It adds support for high bit depth input from Vapoursynth.

Thanks dubhater and 06_taro.

*edit again*

Thanks JEEB, for submitting (http://forum.doom9.org/showpost.php?p=1596309&postcount=426) the patch.

It's a beautiful thing to see people from all over working together. :)

[Myrsloik]

As some you guys probably already know, dubhater started porting a couple of plugins over to VS. You can now download a windows dll for both Histogram (https://github.com/dubhater/vapoursynth-histogram/downloads) and TemporalSoften (https://github.com/dubhater/vapoursynth-temporalsoften/downloads).

*edit*
Right after my post, dubhater updated Histogram. Classic mode works with up to 16 bits now.

Also, 06_taro posted (http://astrataro.wordpress.com/2012/10/27/x264-hi-bpp-y4m/) a special build of x264. It adds support for high bit depth input from Vapoursynth.

Thanks dubhater and 06_taro.

Just going to add some small notes about stuff. Jeeb also deserves credit as he was the one who submitted the actual patch for the extended y4m support to x264.

Dubhater/nodame/jackoneill also deserves more credit for reconstructing temporalsoften's algorithm from inline asm to c. That code can benefit avxsynth too since it's portable.

[Myrsloik]

R14 is out. It's mostly some much needed bugfixing and installer improvements. If you use VSFS it has to be registered again because it's now installed into a slightly different path.

And remember to check out the SDK and try to port your favorite filter from avisynth to help make the project a success.

The bloggy post (http://www.vapoursynth.com/2012/10/r14-improved-packaging/) as usual.

[Mug Funky]

how would one go about opening up temporalsoften to allow arbitrary colourspaces? i gitted the code but remembered as i looked through it that i don't know how to program in C.

[jackoneill]

how would one go about opening up temporalsoften to allow arbitrary colourspaces? i gitted the code but remembered as i looked through it that i don't know how to program in C.
Which ones do you need? Maybe I'll add support. Or maybe not. Who knows...

[Mug Funky]

i was thinking at least on principle it could be adapted to any colourspace, as the native avs one works in rgb and yuv.

specifically, it would be awesome to have 16 bits so i can apply it to linear light and emulate a true motion-blur with it :)

[edit] btw, it goes without saying, but saying is always nicer: thanks for your work on porting from avisynth :) it's an exciting time with vapoursynth at the moment, as it looks like it could really reinvigorate the video-script concept. alas, i need to spend a bit of time learning C, and i don't really have much of that :( i'll share any functions i apply other people's plugins to, of course.

[Myrsloik]

i was thinking at least on principle it could be adapted to any colourspace, as the native avs one works in rgb and yuv.

specifically, it would be awesome to have 16 bits so i can apply it to linear light and emulate a true motion-blur with it :)

[edit] btw, it goes without saying, but saying is always nicer: thanks for your work on porting from avisynth :) it's an exciting time with vapoursynth at the moment, as it looks like it could really reinvigorate the video-script concept. alas, i need to spend a bit of time learning C, and i don't really have much of that :( i'll share any functions i apply other people's plugins to, of course.

Do try to learn c. There are quite a few simple tasks that need to be done too. Such as ports of some of the popular internal avisynth filters or adding override file support to vivtc.
I have quite a few tasks in mind at different difficulty levels for those who want to try their luck or learn something.
Just send me a message here or find me on irc (usually in #avisynth on freenode).

At the moment I'm porting avisource which I'll probably post a first test version of tomorrow.
If there's any interest I'll make the source available for an intermediate step in the porting too. An avisynth avisource with the latest vdub parser. It could be useful to merge back into the avisynth tree... maybe.

[Mug Funky]

if human beings could execute in parallel, and i got myself an upgrade, i'd be on it in a flash.

as it happens, tasks exist in parallel but capability is single-threaded. gotta balance day job, housekeeping, side business, plus baby with one more coming in april :)

though if there's some good learning resources out there that assume no programming knowledge (i fear the visual basic and actionscript i learnt back in the day might have been equivalent to negative learning as far as programming practice goes), i'll jump on them and take my first steps. i figure if programmers can do it, it can't be that hard, but i need to learn in between tasks, which reduces the utility of most noob guides out there.

[JEEB]

though if there's some good learning resources out there that assume no programming knowledge (i fear the visual basic and actionscript i learnt back in the day might have been equivalent to negative learning as far as programming practice goes), i'll jump on them and take my first steps. i figure if programmers can do it, it can't be that hard, but i need to learn in between tasks, which reduces the utility of most noob guides out there.
For C I now dearly recommend Zed A. Shaw's freely available Learn C The Hard Way (http://c.learncodethehardway.org/book/). It gets you right into the usual development cycle with Makefiles, a compiler and valgrind. Unfortunately Valgrind is only for *nix for now, so Windows users won't be able to get its wonderful output.

For editor I kind of recommend Qt's Qt Creator. It does have many advanced features as an IDE, but it lets you "just open files" in it (and edit things rather straightforwardly, which is not something that all high-level development environments can boast with *cough*MSVS*cough*), and has nice header parsing capabilities :) Now if I only could find a way to set it not use CRLF endlines on Windows... ^^;

[mandarinka]

as it happens, tasks exist in parallel but capability is single-threaded. gotta balance day job, housekeeping, side business, plus baby with one more coming in april :)

Ah, just forget programming/open source contributing. Other people can do that /especially since you would have to learn C too = the ones already knowing it can do the job much more efficiently = you will be wasting your time/. Also, in 5-10 years, your code is going to be obsolete and long replaced anyway. Little point in spending precious time on that.

At the same time, nobody is going to substitute you for your kids. IMHO, just leave the unimportant matters to the internets and focus on family :)

[Chirico]

Other people can do that /especially since you would have to learn C too = the ones already knowing it can do the job much more efficiently = you will be wasting your time/.

What a terrible attitude. You could have said that about anyone wanting to learn C for the last 3 decades. Heck, you could say that about learning any skill. There will always be people who are better at some skill than you. Doesn't mean it's a "waste of time" to learn something just because you aren't going to be a guru.

Also, in 5-10 years, your code is going to be obsolete and long replaced anyway. Little point in spending precious time on that.

Yeah, right. Code lives around for decades good or bad. In most cases, people don't want to put in the effort to rewrite or replace things as long as it works. This is true whether it's an open source project or not. One only has to look at the cruft built up in projects like Avisynth, ffdshow, or some the more bizarre and byzantine code sections in ffmpeg/libav that almost no one wants to touch despite people repeatedly talking about how it needs to be rewritten.

[mandarinka]

Look at your examples - Vapoursynth is going to replace Avisynth. FFDshow is likely going to be replaced by LAV Video/Audio. Code in ffmpeg/libav gets rewritten all the time, just not the real arcane parts they are scared of (it's almost as if they focused on polishing the chrome so as to not have free time for the hard parts, haha </troll>).

But that's not important - in the end he has a family. Better if people with free time that is less precious volunteer. (Anyway, we should probably drop this not exactly on-topic discussion).

Edit - (@Chirico)
So basically your point is that he should neglect his children and instead learn programming languages so that he can invest his time into serving your interest in a random video editing software? (Funny how you are requesting that with a moralising tone, though.)

[Chirico]

Look at your examples - Vapoursynth is going to replace Avisynth. FFDshow is likely going to be replaced by LAV Video/Audio.

It's quite unlikely that Avisynth or ffdshow will be fully replaced any time soon. Tons of people will still use them and will continue to do so. And so what if the code, if Mug Funky chooses to write some, gets replaced or rewritten? If he got enjoyment/use out of it along with others why attempt to discourage the endeavor?

Code in ffmpeg/libav gets rewritten all the time, just not the real arcane parts they are scared of (it's almost as if they focused on polishing the chrome so as to not have free time for the hard parts, haha </troll>).

Many parts are constantly being cleaned up and optimized. Hardly any pieces are routinely being rewritten from scratch.

But that's not important - in the end he has a family.

Yeah, and? I have a family, too, and so do plenty of open source volunteers. I'm failing to grasp what your point can possibly be unless you think Mug Funky is not a mature adult who can manage their life properly.

Better if people with free time that is less precious volunteer.

What a bizarre attitude. Volunteering your time to work on hobby projects does not mean your free time is somehow not precious. If the person wants to learn C to do hobbyist development let them do so without being piled on about how it's a waste of their time.

Edit to add: Yes, this is OT and I'll drop it, but it's disheartening to see someone told they shouldn't do something because some random person on the Internet thinks its a waste of that person's time.

[Myrsloik]

Here's a test version of my "port" of avisource. There's really not much left of the original code anymore. It should work ok and be able to open avi, vpy and avs files. There are some issues with v210 and b48r input which will be fixed.

download (https://dl.dropbox.com/u/73468194/avisource.dll)

Usage: core.avisource.AVISource('blah.avi')
Has about the same options as in avisynth

[Mug Funky]

@ JEEB: thanks for the links. i'll check them out when i can. i might make a functional equivalent of temporalsoften as a first project, but written from scratch so i don't have to worry about figuring out which parts are bit-depth specific or are achieved using bitwise trickery. even experienced programmers find other people's code to be gobbledygook...

@ everyone else: lolfest. sorry about the OT derailment - hopefully my next posts will be more useful to the discussion. perhaps i'll teach my son to program in C and learn it while doing so :) chances are he'll be more interested in breaking stuff though.

[edit]

@ Myrsloik: avisource seems to work nicely. cool.

[Robert Martens]

Please excuse me if I'm just looking in the wrong place, but what does everyone recommend for testing a filter's proper function with the various colorspaces offered by Vapoursynth? I've got an Avisynth plugin of mine working in VS with most of the 8 and 16 bit YUV formats, and I can set up the Visual Studio debugger to launch either VirtualDub or MPC-HC (with madVR) to display the output of a test script, but things like Grey16, YUV444P16, and all of the RGB formats are presenting a problem.

Take this script, for example:

import vapoursynth as vs
import sys

core = vs.Core()

core.avs.LoadPlugin(path=r'C:\Program Files (x86)\AviSynth 2.5\plugins\ffms2.dll')

ret = core.avs.FFVideoSource(source=r'DVfulliNTSC.avi', colorspace="YV12")

ret = core.resize.Bicubic(ret, format=vs.RGB24)

last = ret

VirtualDub returns the error "Avisynth open failure: VFW module doesn't support RGB24 output", MPC-HC shows the Vapoursynth error screen (ten seconds of red, green, and blue bars in RGB32), and if I swap the last line of the script with

ret.output(sys.stdout, y4m=False)

and attempt to pipe the script to x264, I get an error from swscaler, "gbrp is not supported as output pixel format". I take it I can't use the resizers' colorspace conversion option to turn my YV12 clip into Planar RGB, but even if I manage to find or generate a test clip that's natively stored in that format, how might I get it into Vapoursynth, through my plugin, and out to a program that can display it?

I've read through the thread, the VS documentation, and some of the source code, but I haven't found anything to tip me off to the right approach here.

[Reel.Deel]

I don't know if this is correct but try using this for RGB output. It works. :)

src = core.resize.Bicubic(src, format=vs.COMPATBGR32)
src = core.std.FlipVertical(src) # When I tried I needed this, not sure if you will too
edit:
I only tried this with VFW.

[Robert Martens]

Ah, damn it, I had it all so clear in my mind I forgot no one else will know what I was trying to do; I'm planning to implement planar RGB processing in my plugin, so I need to get a planar clip from somewhere (in this case by trying to convert a YV12 video), pass it into my plugin, then send it out to some application so I can see if I'm doing everything correctly. The script I posted is pared down from what I had been using, in an attempt to remove my own code from the equation for troubleshooting purposes. I figured I should start by confirming whether or not I can even view planar RGB output in the first place, before worrying about my own processing.

I have tried your suggestion, though, converting to the BGR32 compatibility format before the final output, but I get an access violation from VDub when doing so, the call stack in the report being:

736c2b75: MSVCR100!strncpy [736c0000+2ad0+a5]
6a41600f: vapoursynth!0000600f
6a41d3f5: vapoursynth!_getVapourSynthAPI@4 [6a410000+74d1+5f24]
66a2678c: QtCore4!?start@QThread@@QAEXW4Priority@1@@Z [66a00000+25fc0+7cc]
66aedef9: QtCore4!??0QAbstractEventDispatcher@@IAE@AAVQAbstractEventDispatcherPrivate@@PAVQObject@@@Z [66a00000+edec0+39]
66a237b5: QtCore4!?currentThread@QThread@@SAPAV1@XZ [66a00000+237b0+5]
66a25b49: QtCore4!?setTerminationEnabled@QThread@@KAX_N@Z [66a00000+25b20+29]
66a25f93: QtCore4!?setPriority@QThread@@QAEXW4Priority@1@@Z [66a00000+25c60+333]
7371c556: MSVCR100!_endthreadex [736c0000+5c51c+3a]
7371c600: MSVCR100!_endthreadex [736c0000+5c51c+e4]
751633aa: kernel32!BaseThreadInitThunk [75150000+13398+12]
77039ef2: ntdll!RtlInitializeExceptionChain [77000000+39e8f+63]
77039ec5: ntdll!RtlInitializeExceptionChain [77000000+39e8f+36]

If I remove the initial conversion to planar RGB24, your suggestion works to display the clip as RGB, so I take it that's where the issue lies; I suppose the only answer is to get my hands on a clip that's actually stored as planar RGB, run it through my plugin, and convert to another format on output for preview purposes.

Why I didn't actually try doing that before posting here, I honestly can't say. I guess I'm dumb.

[jackoneill]

i was thinking at least on principle it could be adapted to any colourspace, as the native avs one works in rgb and yuv.

specifically, it would be awesome to have 16 bits so i can apply it to linear light and emulate a true motion-blur with it :)
TemporalSoften now takes up to 16 bits per sample YUV, RGB and Gray. Testing welcome.
Like in the avisynth version, scenechange is not available with RGB input. I'll have to think about that one.

[TheProfileth]

So, I think someone has probably thought this somewhere along the line. When does Vapoursynth get its own sub forum on doom9? Also just wanted to say great job everyone for picking up vapoursynth and running with it, I am sure that by one year's time Vapoursynth will be the predominate non-linear access video filtering suite.

[qyot27]

Since some more options for source filters have arrived, I can now verify that the crash I saw occurring after 99 frames are piped from VapourSynth on Windows is definitely not in my build of FFMS2 (nor was it fixed in the release of R14). I just tested both the MSVC and MinGW builds of the D2VSource filter (http://forum.doom9.org/showthread.php?t=166399), the AVISource port, and even DGDecode.dll itself: the crash still occurs, and at the same 99 frames point.

Would CPU instructions be a possible cause of this? It's been the one I've wondered about for a while, since my processor doesn't have SSE2 and that seems to be the most common issue that arises when it's instruction set related. Dunno why it would suddenly kick up a fuss at ~100 frames in, though. If the build system optimizes for this during the compilation process, that could explain why my Linux builds of VapourSynth don't exhibit it (unless it's actually some kind of Windows-specific problem that just happens to get exposed because of my ancient hardware).

[Reel.Deel]

@ Myrsloik
I was testing out AVIsource with a 4:2:0 UTVideo compressed source and I noticed it was decompressing to YV16. I added the pixel_type="YV12" and VDub gives me the following error:
Avisynth open failure:
Python exception: "AVISource: the video decompressor couldn't produce YV21 output"
Also if I change pixel_type to YV16, the error is ...."couldn't produce YV61 output"

This is the script (vpy) I used:
import vapoursynth as vs
core = vs.Core()
core.std.LoadPlugin(path=r'C:\Vapoursynth\avisource.dll')
src = core.avisource.AVISource(path=r'D:\TS.avi', pixel_type="YV12", fourcc="ULY0")
last = src

[Myrsloik]

I typoed the fourccs for yv12 and yv16. Get an updated dll here (https://dl.dropbox.com/u/73468194/avisource.dll). (completely untested but at least the code is less wrong now)

[hajj_3]

there is a new build of x264 out (v2230), there is a change in there for vapoursynth:

commit 28ddb0dd533154b58f9147932fb1dec4c74127c8 r2226
Author: Jan Ekström <jeebjp@gmail.com>
Date: Sun Oct 7 21:12:05 2012 +0300

Add support for the ffmpeg/vapoursynth high bit depth y4m extensions

[Mug Funky]

i'm having an issue with building luts for use with... Lut.

anything over 8 bits is taking a phenomenally long time to build. is it a python limitation or a bug somewhere?

check:

ret = core.resize.Lanczos(clip=ret,format=vs.YUV422P16)

luty = []
for x in range(2**ret.format.bits_per_sample):
x = float(x / 2**ret.format.bits_per_sample)
x = 10 ** x
x = x - 1
x = x * (.112 * 2**ret.format.bits_per_sample)
x = int(x + .5)
x = max(min(x,2**ret.format.bits_per_sample - 1),0)
luty.append(x)

ret = core.std.Lut(clip=ret,lut=luty,planes=0)

there's source stuff there of course - this is the relevant snippet. when the resize line is set to YUV422P8 all is fine, but P10 or P16 seem to never give an output (i've had vdub open for 10 minutes on a 16 bit one). 65536 values doesn't seem too big to me.

btw, this is an attempt to undo a lin-to-log, eventually apply a temporalsoften, then apply a lin-to-log to allow me to output 10 bit motion-blurred log files from an 8-bit log source (a canon 550D set to 720p60). if i can make it work, then i can almost shoot with the dynamic range of film on my cheapie DSLR...

[StainlessS]

I dont understand Python but it might be a bit quicker if you precalculated eg "2**ret.format.bits_per_sample" rather than doing it time and time
again.
Also, "luty.append(x)" looks like it is doing a lot of reallocating memory, copying existing contents and then adding another. (unless is linked list).

EDIT: Perhaps something like:


SZ=2**ret.format.bits_per_sample
Q=.112 * SZ
Z=SZ-1
luty = [SZ]
for i in range(SZ): # Maybe SZ should be Z here
x = float(i / SZ) # Maybe SZ should be Z here
x = 10 ** x
x = x - 1
x = x * Q
x = int(x + .5)
x = max(min(x,Z),0)
luty[i]=x

ret = core.std.Lut(clip=ret,lut=luty,planes=0)


You've used x as both for loop index and as a temp variable, hence the main problem, where would loop 4E4 (forever).

[Chikuzen]

I tried as follows

>>> import vapoursynth as vs
>>> core = vs.Core()
>>> core.std.LoadPlugin('G:/vsplugins/vsrawsource.dll')
>>> clip = core.raws.Source('D:/source/sample.y4m')
>>> clip = core.resize.Point(clip, format=vs.YUV420P16)
>>> luty = [x for x in range(2 ** 16)]
>>> clip = core.std.Lut(clip, luty, 0)

and Python shell was freezed.
so, this may be a bug.


Also, "luty.append(x)" looks like it is doing a lot of reallocating memory, copying existing contents and then adding another. (unless is linked list).
This is nonsence.
this processing does not take 1 second in total.

[Myrsloik]

i'm having an issue with building luts for use with... Lut.

anything over 8 bits is taking a phenomenally long time to build. is it a python limitation or a bug somewhere?

check:

ret = core.resize.Lanczos(clip=ret,format=vs.YUV422P16)

luty = []
for x in range(2**ret.format.bits_per_sample):
x = float(x / 2**ret.format.bits_per_sample)
x = 10 ** x
x = x - 1
x = x * (.112 * 2**ret.format.bits_per_sample)
x = int(x + .5)
x = max(min(x,2**ret.format.bits_per_sample - 1),0)
luty.append(x)

ret = core.std.Lut(clip=ret,lut=luty,planes=0)

there's source stuff there of course - this is the relevant snippet. when the resize line is set to YUV422P8 all is fine, but P10 or P16 seem to never give an output (i've had vdub open for 10 minutes on a 16 bit one). 65536 values doesn't seem too big to me.

btw, this is an attempt to undo a lin-to-log, eventually apply a temporalsoften, then apply a lin-to-log to allow me to output 10 bit motion-blurred log files from an 8-bit log source (a canon 550D set to 720p60). if i can make it work, then i can almost shoot with the dynamic range of film on my cheapie DSLR...

I found the typo in the 9-16 bit code. It entered an infinite loop. Now it takes under a second for a 16bit lut on my computer.

[StainlessS]

EDIT: Perhaps something like:


SZ=2**ret.format.bits_per_sample
Q=.112 * SZ
Z=SZ-1
luty = [SZ]
for i in range(SZ): # Maybe SZ should be Z here
x = float(i / SZ) # Maybe SZ should be Z here
x = 10 ** x
x = x - 1
x = x * Q
x = int(x + .5)
x = max(min(x,Z),0)
luty[i]=x

ret = core.std.Lut(clip=ret,lut=luty,planes=0)


You've used x as both for loop index and as a temp variable, hence the main problem, where would loop 4E4 (forever).

Probably as already pointed out.

[TheFluff]

Also, "luty.append(x)" looks like it is doing a lot of reallocating memory, copying existing contents and then adding another. (unless is linked list).
wrong

You've used x as both for loop index and as a temp variable, hence the main problem, where would loop 4E4 (forever).
That's not how Python works. "for x in range()" iteratively sets x to each of the values in the list returned by range(), it's not a c-style for. It's completely safe to manipulate the loop variable inside the loop. Changing the loop variable does not change the list, either.

Example:
>>> for x in range(0,4):
... x = x+2
... print x
...
2
3
4
5

As Myrsloik said, it was a bug in vapoursynth.

[StainlessS]

Thanks for your explanation Fluffy, did look like an infinite loop to me.
As it happens Iv'e just downloaded VapourSynth & Python 3.3 a couple of moments ago so Ill have something new to play with.

[Mug Funky]

ah, excellent. being a python novice, i just copied the example lut in the vapoursynth docs and hacked at it.

seeing it work in 8 bit suggested to me it was not something i was doing.

i will precalculate the bit depth thing though - save a couple of instructions here and there.

[Myrsloik]

With the official binary of R13 I'm consistently getting a crash after frame #99. It affects all the files I've tested it with (tested with .avi, .mp4, .mov, .mkv, and .flv files...still results in x264 only getting 99 frames to encode before crashing). The Windows error reporting dialog says the crash is in vapoursynth.dll, so is it safe to say that's where it really is? Namely because the script only uses FFMS2, so if it was actually in the source filter I'd think it would report that ffms2.dll crashed. I tried culling what little information gdb can generate without debug symbols, but that was completely fruitless (tried hooking both python itself and x264...neither one gave me anything to go on).

None of this happens on Linux, so it seems like it might just be a problem with compilation settings or something (I'll have to try satisfying the dependencies first to attempt building under Windows, though). Unless this is part of a fix that occurred between the release of R13 and HEAD.

Is there any kind of internal error reporting type of thing that could be used to help diagnose this?


Some output from the Command Prompt:
>python vpytestscript.vpy | x264 --stdin y4m --preset ultrafast --crf 18 -o testvs.mkv -
y4m [info]: 1920x800p 0:0 @ 24000/1001 fps (cfr)
y4m [info]: color matrix: undef
x264 [info]: using cpu capabilities: MMX2 Cache32
x264 [info]: profile Constrained Baseline, level 4.0
x264 [info]: started at Sat Oct 27 18:04:29 2012

x264 [info]: frame I:1 Avg QP:15.00 size: 92219
x264 [info]: frame P:98 Avg QP: 7.36 size: 891
x264 [info]: mb I I16..4: 100.0% 0.0% 0.0%
x264 [info]: mb P I16..4: 0.0% 0.0% 0.0% P16..4: 1.9% 0.0% 0.0% 0.0% 0.0% skip:98.1%
x264 [info]: coded y,uvDC,uvAC intra: 11.2% 10.7% 10.7% inter: 0.6% 0.4% 0.4%
x264 [info]: i16 v,h,dc,p: 92% 6% 2% 0%
x264 [info]: i8c dc,h,v,p: 89% 7% 3% 0%
x264 [info]: kb/s:347.87

encoded 99 frames, 2.540 fps, 349.09 kb/s, 175.96 KB
x264 [info]: ended at Sat Oct 27 18:05:08 2012
x264 [info]: encoding duration 0:00:39

And one with the normal AviSynth side of the .dll to prove there's nothing wrong there. I killed it at 151 frames so I wouldn't have to wait around too long.
>x264 --preset ultrafast --crf 18 -o testvs-avs.mkv testvs.avs
avs [info]: 1920x800p 0:0 @ 24000/1001 fps (cfr)
avs [info]: color matrix: undef
x264 [info]: using cpu capabilities: MMX2 Cache32
x264 [info]: profile Constrained Baseline, level 4.0
x264 [info]: started at Sat Oct 27 18:05:54 2012
[5.0%] 151/3008 frames, 5.969 fps, 249.76 kb/s, 192.02 KB, eta 0:07:58, est.size

x264 [info]: frame I:1 Avg QP:15.00 size: 92219
x264 [info]: frame P:150 Avg QP: 7.27 size: 692
x264 [info]: mb I I16..4: 100.0% 0.0% 0.0%
x264 [info]: mb P I16..4: 0.7% 0.0% 0.0% P16..4: 1.5% 0.0% 0.0% 0.0% 0.0% skip:97.9%
x264 [info]: coded y,uvDC,uvAC intra: 5.7% 5.4% 5.3% inter: 0.5% 0.3% 0.2%
x264 [info]: i16 v,h,dc,p: 95% 3% 2% 0%
x264 [info]: i8c dc,h,v,p: 95% 4% 2% 0%
x264 [info]: kb/s:248.96

aborted at input frame 151, output frame 151
encoded 151 frames, 5.969 fps, 249.76 kb/s, 192.02 KB
x264 [info]: ended at Sat Oct 27 18:06:20 2012
x264 [info]: encoding duration 0:00:26

>

I figured out why. Sse2 math is enabled in my compiles. There's an event every 100 frames or so when internal statistics are updated using floating point.
So what can I say, vapoursynth requires a cpu with sse2. This is to make things easier for plugin writers too. Nowadays sse2 support is about as widespread as pure mmx support was 10 years ago or so.

[Keiyakusha]

Hi, I have few questions.

import vapoursynth as vs
core = vs.Core()
ret=core.std.BlankClip(format=vs.YUV420P8,color=[255,255,255]) #Is this supposed to be purple color? And color=[0,0,0] is green, I kind of expected black...
ret=core.std.CropRel(ret,right=638)
ret=core.resize.Point(ret,format=vs.YUV444P8) # this gives crashes/hangs - access violation in msvcr100
last=ret

I'm using VS r14, vfw module. Am I doing something wrong?

[Myrsloik]

Hi, I have few questions.

import vapoursynth as vs
core = vs.Core()
ret=core.std.BlankClip(format=vs.YUV420P8,color=[255,255,255]) #Is this supposed to be purple color? And color=[0,0,0] is green, I kind of expected black...
ret=core.std.CropRel(ret,right=638)
ret=core.resize.Point(ret,format=vs.YUV444P8) # this gives crashes/hangs - access violation in msvcr100
last=ret

I'm using VS r14, vfw module. Am I doing something wrong?

For color you're setting the raw pixel values so for yuv 255, 128, 128 should give you white which probably is what you expected. The resize thing I'll have to test and see what happens...

[Chikuzen]

>>> import vapoursynth as vs
>>> core = vs.Core()
>>> clip = core.std.BlankClip(format=vs.YUV420P8, color=[255, 128, 128])
>>> clip = core.std.CropRel(clip, right=638)
>>> clip = core.resize.Point(clip, format=vs.YUV444P8)
>>> print(clip)
VideoNode
Format: YUV444P8
Width: 2
Height: 480
Num Frames: 240
FPS Num: 24
FPS Den: 1
Flags: No Cache

>>> out = open('yuv444p8_width2.raw', 'wb')
>>> clip.output(out)
[swscaler @ 02979cc0] 2x480 -> 2x480 is invalid scaling dimension
[swscaler @ 02979cc0] 2x480 -> 2x480 is invalid scaling dimension
[swscaler @ 02979cc0] 2x480 -> 2x480 is invalid scaling dimension
[swscaler @ 02979cc0] 2x480 -> 2x480 is invalid scaling dimension
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "vapoursynthpp.pyx", line 671, in vapoursynth.VideoNode.output (src\cython\vapoursynthpp.c:10473)
vapoursynth.Error: 'Failed to retrieve frame 1 with error: Resize: context creation failed'

seems it's swscale's limitation (or bug).
however, VS has some problem on error handling too.

[Myrsloik]

seems it's swscale's limitation (or bug).

Yes, swscale usually fails below 32x32 sized output and probably input too. It's simply useless and broken. Other bugs are green tint, some really broken resizer modes that mess up chroma completely. The inability to both convert to and from planar rgb, respecting the color matrix and most other things. I hope to replace it soon because it sucks.

Prodding the developers has given exactly 0 improvements over a period of 6 months. Or 6 years depending on how you count...

[Chikuzen]

Other bugs are green tint, some really broken resizer modes that mess up chroma completely.
I heard that issue was already fixed, isn't it?
https://bugzilla.libav.org/show_bug.cgi?id=33

[kolak]

Green tint is there for sure, but it happens in specific cases- eg yuv42210bit to 8 bit is fine, but yuv44410bit to 8bit is not. In the same time yuv44410bit to 422 10bit is fine also- mess :)

[Mug Funky]

resampleHQ might be a good place to look for a better resizer core. works in linear light, too, which i reckon could be broken out as a separate special colour space.

[Myrsloik]

resampleHQ might be a good place to look for a better resizer core. works in linear light, too, which i reckon could be broken out as a separate special colour space.

There is already a much better one in the works by cretindesalpes.

[Adub]

Myrsloik,

I'm interested in using Vapoursynth as a basis for some academic research I'm conducting. TheFluff encouraged me to check out this project over Avxsynth, as he said Avisynth's caching mechanism "is an incredible mess." source (http://forum.doom9.org/showpost.php?p=1599555&postcount=141)

The short version is that I would like to port a few C versions of filters over to CUDA and evaluate the performance difference. The greatest performance gains can be established if we can ship a large number of frames across the PCI-Express bus in one fell swoop, perform the necessary computations (ideally, chaining several filters together, as that will let us keep the frames on the card) and then ship the frames back.

Now, do you think that there is a path to this in the current implementation of Vapoursynth? For instance, when a filter requests a frame using requestFrameFilter(), this could trigger a transfer of frames across the bus? Or is there another implementation that you have in mind?

I plan to do most of my development in Linux (for now), as I will be gaining access to several workstations running new Kepler cards, and my thesis advisor wants to run these workstations on Arch Linux for now. I have an Ubuntu rig at home for testing with lower end cards as well.

I look forward to your input on this, as I'd like to have as little data transfer overhead as possible.

[Myrsloik]

This is territory I didn't plan to seriously explore for quite a few more months. If you want to try to hack the stuff in I'd suggest something like this.

Modify the VSFrame class so it represents either a local or a gpu stored frame. Keep the total memory usage counters separated of course.

Then simply write a filter that moves frames to the gpu. I don't know what the mapping looks like here in cuda. Add a filter that does the gpu to local memory transfer too.

After that just write your gpu filters and see if it works. So sure, you can probably hack something in fairly easily if you really want to.


And now something completely unrelated. .. I've posted a last with all the tasks I could think of that would be good for VapourSynth at my blog. Tasks (http://www.vapoursynth.com/2012/11/vapoursynth-tasks/)

[Myrsloik]

Hi Myrsloik,

Just want to know do you have any clue to this problem (http://forum.doom9.org/showthread.php?p=1599044#post1599044)? Thanks.

It's a bug in mvtools. For some reason it doesn't use the emms instruction to restore the fpu state before returning when pel=1. Contact the mvtools author to get it fixed. I will never add a workaround for this in vs because it really is a fatal error.

[Myrsloik]

R15 has come. It adds some filters and some fixes but mostly it's internal changes to have more efficient memory handling. See the first post for the changelog. I also recommend looking at the python documentation as it now documents the most important classes and all their methods and attributes.

The usual blog post contains details on how to got from API R2 to R3:
R15 - API Improvements (http://www.vapoursynth.com/2012/11/r15-api-improvements/)

[Mr VacBob]

There is already a much better one in the works by cretindesalpes.

What's much better about it?

[TheFluff]

What's much better about it?

this post makes me wonder if you're trolling or if you're actually a genuine swscale apologist

the latter possibility is quite unsettling

[Mr VacBob]

Oh, I can think of lots of things wrong with it. It's unreadable, single-threaded, could use GPGPU (or anything newer than MMX), missing 10-bit+ and linear light processing, and could use an EEDI scaler. And the Gaussian filter didn't work last time I tried.

I have a new (NIH) resizer project in progress myself, but it's just for thumbnailing and not really worth trying to do much else with it.

Which ones did you want to address?

[Myrsloik]

Oh, I can think of lots of things wrong with it. It's unreadable, single-threaded, could use GPGPU (or anything newer than MMX), missing 10-bit+ and linear light processing, and could use an EEDI scaler. And the Gaussian filter didn't work last time I tried.

I have a new (NIH) resizer project in progress myself, but it's just for thumbnailing and not really worth trying to do much else with it.

Which ones did you want to address?

All of them would be nice. I could live without the gpu stuff though.

[Chikuzen]

@Myrsloik
How about this patch (http://forum.doom9.org/showpost.php?p=1600823&postcount=6) ?

[Adub]

This is territory I didn't plan to seriously explore for quite a few more months. If you want to try to hack the stuff in I'd suggest something like this.

Modify the VSFrame class so it represents either a local or a gpu stored frame. Keep the total memory usage counters separated of course.

Then simply write a filter that moves frames to the gpu. I don't know what the mapping looks like here in cuda. Add a filter that does the gpu to local memory transfer too.

After that just write your gpu filters and see if it works. So sure, you can probably hack something in fairly easily if you really want to.
Tasks (http://www.vapoursynth.com/2012/11/vapoursynth-tasks/)

Okay, I'll see what I can do with this. I am worried a bit about blocking, but I'll see what we can do with this method. I won't be able to work on it for a few weeks, as I have other projects for classes to get done, but I should be able to make something work over break.

[Chikuzen]

How can I read AVI file containing alpha with avisource?
I tried

>>> import vapoursynth as vs
>>> core = vs.Core()
>>> print(core.version())
VapourSynth Video Processing Library
Copyright (c) 2012 Fredrik Mellbin
Core r15
API r3

>>> core.std.LoadPlugin('G:/vsplugins/avisource.dll')
>>> clip = core.avisource.AVISource('320x240_uly0.avi')
>>> print(clip)
VideoNode
Format: YUV420P8
Width: 320
Height: 240
Num Frames: 600
FPS Num: 60
FPS Den: 1
Flags: No Cache

>>> clip = core.avisource.AVISource('alpha_test.avi')

and got crash.

sample (http://www.mediafire.com/download.php?9c4d6x05d908dxp) and information (http://pastebin.com/0kBM2PY7)

[kolak]

AVIsource does not seams to have any RGBA color space implemented yet.

[Myrsloik]

How can I read AVI file containing alpha with avisource?
I tried
...

sample (http://www.mediafire.com/download.php?9c4d6x05d908dxp) and information (http://pastebin.com/0kBM2PY7)

I'll investigate it. The avisource stuff still is still alpha quality, I just wanted to get it out so people would send me samples of stuff that doesn't work.

[Myrsloik]

After upgrading to r15, MDegrain crashed. :scared:
http://img7.ressim.net/out.php/i6363580_mdegrain.png

Found it. The avisynth compat stuff is broken for everything except yv12 due to some stricter checks I added. I'll release R16 soon, just want to poke avisource a bit since it also has issues.

[osgZach]

Wow, neat thread.. Neat tool (I hope)..

There are 26 pages I truly can't be bothered to read, so I'm sorry if this has been asked before (linking to answers in a previous post is quite fine)

Is this intended to replace AVIsynth (eventually?) Are there long-term development plans? (i.e its not going to become vaporware because someone gets bored) Has serious planning already been done for development that would put this at the current level of Avisynth functionality / speed, but also be able to surpass it as future development continues?

I thought I saw cross-platform mentioned.. Is this the case? (Windows/Linux/Apple is a quite attractive trifecta)

Also, any plans to develop a front-end type program for this? ala Vdub, etc.. I know we had an active project for a Python based Avisynth front-end/editing application, so that could always prove a valuable base to start with..

[Myrsloik]

Wow, neat thread.. Neat tool (I hope)..

There are 26 pages I truly can't be bothered to read, so I'm sorry if this has been asked before (linking to answers in a previous post is quite fine)

Is this intended to replace AVIsynth (eventually?) Are there long-term development plans? (i.e its not going to become vaporware because someone gets bored) Has serious planning already been done for development that would put this at the current level of Avisynth functionality / speed, but also be able to surpass it as future development continues?

I thought I saw cross-platform mentioned.. Is this the case? (Windows/Linux/Apple is a quite attractive trifecta)

Welcome to my 26th page summary post!
Is it intended to replace avisynth?
YES! But it may take another 3 months before all really useful filters are ported (mvtools excluded since it's a huge and overly complicated mess)
Is it really cross platform?
Yes, many of the people who give me the most feedback are linux users. OSX supposedly also works but none of those users report their results to me.
Speed?
Since it can usually use all cores at once it surpasses plain avisynth for speed and gets close to avisynth-mt at its best, minus the crashes.
Serious planning?
Will some papers full of notes about what needs to be done do? there's also a suggest task list for contributors. Mostly it's bug fixing now and porting more useful plugins so the step to 64bit will be less painful.
Long term development plans?
Yes, I guess so, doesn't look like I'll find a job before Christmas at least.

Since it's binary compatible with most existing avisynth plugins you can use it for quite a bit already.

[osgZach]

Thanks for taking the time to respond to my question.

I really hope you have great success with this project, and we can reach a point where we blow past Avisynth in terms of functionality and other limiting factors (speed /technology etc).
I will be watching this software with great interest..

Maybe this is a bit of a stretch, but would it be right to assume, that since this appears to be built on Python, that all those neat libraries for mathematical and scientific calculations would be very useful for developers writing new plugins for this system? It would be really awesome to get the benefits of such libraries, I am thinking from a speed / complex operations standpoint.. Perhaps we will end up with new filters that not only replicate previous Avisynth filters, but do it better, faster, and more accurate / go beyond what the originals would be capable of in a reasonable amount of processing time...

It would be really nice to see some GPU agnostic assisted plugins come out of this, that prove to be useful and offer significant advantages.

[Myrsloik]

GPU stuff is overhyped. Proper coding and common sense will get you almost as far for most filters. I'll probably add support for it in some form one day anyway just to prove that I'm right.
A few facts about python. It's really slow and in practice single threaded. The c libraries behind the calculation modules will run much faster if used directly in a c plugin.
Maybe it could be usable for filter prototyping for some people but I predict a huge yawn there. Video filters also aren't really "scientific calculations", they're more of an exercise in stuffing as many pixels into an xmm register as possible.

[Myrsloik]

merge is slow, uses less than 40% CPU.

Post the whole script and define slow.

[Myrsloik]

38% CPU usage
1~ FPS output

And now the script.

[ryrynz]

Are there any plans for a DirectShow filter that can serve as a replacement for ffdshow's AviSynth processing?


Not at this time. It'd be pretty easy to implement for someone who is familiar with dshow though (and that's usually the bottleneck for dshow-related stuff anyway).

Anyone able to poke around for someone capable of doing this?

[Chikuzen]

Could std.AddBorders add a 'color' argument to be able to specify the color(like std.BlankClip does)?

I think it should be added.
addborders2.dll (http://www.mediafire.com/download.php?2nw4k3vtzg4b4nb)
http://pastebin.com/YnhJWvKu

[Myrsloik]

Could std.AddBorders add a 'color' argument to be able to specify the color(like std.BlankClip does)?

I'll add it in the next version.

[jackoneill]

I think it should be added.
addborders2.dll (http://www.mediafire.com/download.php?2nw4k3vtzg4b4nb)
http://pastebin.com/YnhJWvKu

I'm curious... how big is a wchar_t on your system?

#include <stdio.h>
#include <wchar.h>

int main() {
printf("%zu\n", sizeof(wchar_t));
return 0;
}

That prints "4" here.

Well, in any case, if I read your code right, it only works correctly if bytesPerSample is the same as sizeof(wchar_t).

[Chikuzen]

I'm curious... how big is a wchar_t on your system?


hmm, stddef.h(mingw) says "/* wchar_t is unsigned short for compatibility with MS runtime */"

my code may be have problem on 64bit Linux/Mac:(

[TheFluff]

wchar_t is two bytes on win32 (UTF-16, argh) but IIRC the spec only says it has to be "at least as wide as char" so it can really be anything from 1 byte and up. As jackoneill points out, it's 4 bytes on many Linux systems.

[Chikuzen]

wchar_t is two bytes on win32 (UTF-16, argh) but IIRC the spec only says it has to be "at least as wide as char" so it can really be anything from 1 byte and up. As jackoneill points out, it's 4 bytes on many Linux systems.

yeah, I confirmed it on Linux.
Fix version http://pastebin.com/aRFGAka3

[sneaker_ger]

Is there a way to prevent this mess when viewing the function list?
http://www.abload.de/thumb/function_list_mess5to2g.png (http://www.abload.de/img/function_list_mess5to2g.png)

[Chikuzen]

Is there a way to prevent this mess when viewing the function list?


#vsfunclist.py
import subprocess

def show(core):
fname = 'funclist.txt'
file = open(fname, 'w')
file.write(core.list_functions())
file.close()
subprocess.Popen(['notepad', fname])



>>> import vapoursynth as vs
>>> import vsfunclist as fl
>>> core = vs.Core()
>>> fi.show(core)


Python is very powerful language :D

[Myrsloik]

Is there a way to prevent this mess when viewing the function list?
http://www.abload.de/thumb/function_list_mess5to2g.png (http://www.abload.de/img/function_list_mess5to2g.png)

Ask for all functions, get all functions.

Doing something like core.avs.list_functions() also works if you only want to view one plugin. The interactive python window you can open from the start menu has nicer line breaks too.

[Myrsloik]

import vapoursynth as vs
core = vs.Core()
core.std.LoadPlugin(path_to_ffms2)
clip1 = core.ffms2.Source(path_to_file)#720p24
clip2 = core.std.Reverse(clip1)
last = core.std.Merge([clip1,clip2],0.5)

#output is from 1 to 5 FPS depending on content, CPU usage mostly at 45%
#import os
#with open(os.devnull,"wb") as null:
# last.output(null)


It's not merge that makes your script slow. It's what you're doing to the poor source filter.

1. Reverse() will always be slow since decoding a file backwards is slow.
2. You're pulling both inputs from the same source, it provokes it to seek to first frame, last frame, first frame + 1, last frame - 1, this is horribly slow. Using two instances of the source filter, one for the reversed clip and one for the forward one will most likely be a lot faster.
3. Because of 1 this will never be a fast operation, reverse could be optimized slightly by caching chunks at a time I guess but that definitely makes it a bit more complicated to implement and quite ram hungry too.

[Myrsloik]

R16 release to fix the major breaking issues in R15.
R16 – The Slightly Less Broken Release (http://www.vapoursynth.com/2012/11/r16-the-slightly-less-broken-release/)

[qyot27]

I figured out why. Sse2 math is enabled in my compiles. There's an event every 100 frames or so when internal statistics are updated using floating point.
So what can I say, vapoursynth requires a cpu with sse2. This is to make things easier for plugin writers too. Nowadays sse2 support is about as widespread as pure mmx support was 10 years ago or so.
Granted r337 prevents using it at all now on my old Celeron Coppermine, but I hadn't experienced any crashing when using native Linux builds; no problem at all serving out >100 frames there. Same computer, so it was still only restricted to the instruction sets the Pentium III had.

Would it have eventually had a problem, though, given enough frames? Or if it was simply using MMX/SSE instead, was/is there something unstable about it on a basic level that requires SSE2 (or disabling asm entirely, if it came to that) to solve?

[jackoneill]

Granted r337 prevents using it at all now on my old Celeron Coppermine, but I hadn't experienced any crashing when using native Linux builds; no problem at all serving out >100 frames there. Same computer, so it was still only restricted to the instruction sets the Pentium III had.

Would it have eventually had a problem, though, given enough frames? Or if it was simply using MMX/SSE instead, was/is there something unstable about it on a basic level that requires SSE2 (or disabling asm entirely, if it came to that) to solve?

You compiled the Linux version on the machine in question, didn't you? I imagine the compiler wouldn't generate code that can't run there, would it?

[sneaker_ger]

Since upgrading from r15 to r16 I've been experiencing random crashes when loading in VirtualDub through VfW. (Either crashes directly when loading the script or sometimes when skipping to a random frame)
Anyone else experiencing this? What more info would you need to find the problem?

http://pastebin.com/08qKfLqB (VirtualDub crash report)

[Myrsloik]

We've had SSE2 since forever. First CPU with it released 12! YEARS! AGO! Since then most cpus have had it, a few poor early athlons excluded. SSE2 is also required on any x64 cpus which have dominated the sales for a loooooong time.

We've had x64 cpus for 9 years.

I don't know what else to say, just go read my minimum requirements blog post. I have no intention of making the lives of plugin writers more difficult just to support a few ancient computers out there. Because in the end this project depends more on being reasonable to develop for than pentium 3 support.

[Myrsloik]

Since upgrading from r15 to r16 I've been experiencing random crashes when loading in VirtualDub through VfW. (Either crashes directly when loading the script or sometimes when skipping to a random frame)
Anyone else experiencing this? What more info would you need to find the problem?

http://pastebin.com/08qKfLqB (VirtualDub crash report)

The crash report looks like it's just pointing at random memory and the stack trace is weird too.

I need something else to go on. I've seeked around in vfw a bit and it doesn't seem to crash here. Maybe someone else has the same issue?
Does it go away when you use a simpler script?

[Robert Martens]

I'm having what looks like the same issue, a std::bad_alloc exception thrown from KernelBase when my filter attempts to copy part of one frame to another, but only when I use VSHelper.h's vs_bitblt. When I loop through the pixels myself, copying things manually, everything works as expected.

The code is a bit sloppy at the moment while I add Vapoursynth support, but if it'd help you track down the problem I can pack something up and send it along.

[Myrsloik]

I'm having what looks like the same issue, a std::bad_alloc exception thrown from KernelBase when my filter attempts to copy part of one frame to another, but only when I use VSHelper.h's vs_bitblt. When I loop through the pixels myself, copying things manually, everything works as expected.

The code is a bit sloppy at the moment while I add Vapoursynth support, but if it'd help you track down the problem I can pack something up and send it along.

Yes, I definitely need to see some code to figure out why it's happening. It's also odd that it throws a std::bad_alloc exception every time, are you accidentally writing to the wrong destination?

[sneaker_ger]

Can't really make it any simpler, because I need to convert 16 PNG input to 8 bit YUV to even make it work with VfW output.
import vapoursynth as vs
import sys
import os

core = vs.Core()

core.std.LoadPlugin(path=r'C:\vsimagereader.dll')
ext = '.png'
dir = 'C:/sintel-4k-png16/'
srcs = [dir + src for src in os.listdir(dir) if src.endswith(ext)]
ret = core.imgr.Read(srcs, fpsnum=24, fpsden=1)

core.std.LoadPlugin(r'C:\fmtconv.dll')

ret = core.fmtc.matrix(clip=ret, mat="709", col_fam=vs.YUV)
ret = core.fmtc.bitdepth(clip=ret, bitdepth=8)

last = ret

Setting core threads to 1 greatly reduces the number of crashes. (From like 3 : 1 on opening the script to 15 : 1)

[sneaker_ger]

Also noticed that VirtualDub does not free up all the memory when closing the script, so maybe it just runs out of memory. (And earlier with more threads)

[Robert Martens]

Yes, I definitely need to see some code to figure out why it's happening. It's also odd that it throws a std::bad_alloc exception every time, are you accidentally writing to the wrong destination?

Probably something likewise idiotic, and I wouldn't be surprised or offended to find at least some portion of the code highlighted in a future Vapoursynth blog entry. I'd have kept my mouth shut and spent at least a few days combing over my work before saying anything, if not for sneaker_ger's crash report mentioning KernelBase and bad_alloc.

Give me a few minutes to pack things up for you and I'll upload the source to Mediafire.

[qyot27]

You compiled the Linux version on the machine in question, didn't you? I imagine the compiler wouldn't generate code that can't run there, would it?
Yes I did, and that's generally what I figured was happening as well. My attempts at trying to get a Windows build cross-compiled (or natively compiled under either GCC-MSys or MSVC-MSys) all failed at one point or another, or resulted in a borked .dll.


At this point I can live with it (especially since I do have access to a Core 2, Athlon64, and Core i5, just not constantly), I'm just more curious about the architectural design.

[Robert Martens]

All right, here goes nothing: http://www.mediafire.com/?vzs4p8fauvdaevb

The plugin is the next version of TurnsTile (http://forum.doom9.org/showthread.php?t=158695), highlighted here: http://www.youtube.com/watch?v=WFJreg-LW50

I've stripped out everything related to both the Avisynth/Avxsynth version of the plugin and my use of CMake, and made a new project with only Vapoursynth-relevant content. The zip file includes the source and headers, a Visual Studio solution, a ten frame test clip with associated test tilesheet, and a .vpy script that's already set up to load the clips and call my function (though you'll need to change the script's absolute paths to the plugin DLL and test footage, of course). I've tested the zip file myself, unpacking it to a new directory and building the plugin, and it reliably demonstrates the crash.

You shouldn't need to bother with any of the function's other parameters, as the test script provokes the problem as is. Set the VS debugger to run the latest experimental release of VirtualDub with the provided test script as the command argument, and you should be set. I've marked portions of the code with "Myrsloik" in four places, so just search for your username and you'll get to the sections I'm having trouble with. The package as is has the working loops commented out, immediately below the broken version, so switching back and forth should be easy enough.

Let me know if there's anything else I can do to help.

[Keiyakusha]

I need to convert 16 PNG input to 8 bit YUV to even make it work with VfW output.
You probably want to say "even make it work in VDub"
vfw supports P016, Vdub is not.

Edit: i mean not only p016, vfw module supports more than that

[Myrsloik]

All right, here goes nothing: http://www.mediafire.com/?vzs4p8fauvdaevb

The plugin is the next version of TurnsTile (http://forum.doom9.org/showthread.php?t=158695), highlighted here: http://www.youtube.com/watch?v=WFJreg-LW50

I've stripped out everything related to both the Avisynth/Avxsynth version of the plugin and my use of CMake, and made a new project with only Vapoursynth-relevant content. The zip file includes the source and headers, a Visual Studio solution, a ten frame test clip with associated test tilesheet, and a .vpy script that's already set up to load the clips and call my function (though you'll need to change the script's absolute paths to the plugin DLL and test footage, of course). I've tested the zip file myself, unpacking it to a new directory and building the plugin, and it reliably demonstrates the crash.

You shouldn't need to bother with any of the function's other parameters, as the test script provokes the problem as is. Set the VS debugger to run the latest experimental release of VirtualDub with the provided test script as the command argument, and you should be set. I've marked portions of the code with "Myrsloik" in four places, so just search for your username and you'll get to the sections I'm having trouble with. The package as is has the working loops commented out, immediately below the broken version, so switching back and forth should be easy enough.

Let me know if there's anything else I can do to help.

From a quick glance on my phone I'm wondering why you don't use tileW_UV in the bitblt. I'll compile and debug it properly tomorrow if that isn't the real issue.

[Robert Martens]

I didn't use tileW_UV for the blit because I was rushing, and didn't spend enough time looking over my code. I wrote the planar RGB version first, where I use the same numbers for each call to vs_bitblt, and then did the YUV version without thinking. Switching to use tileW_UV and tileH_UV solves the issue for me. Thanks for catching that, and sorry to sneaker_ger that my mistake isn't related to your crash.

While I'm here, I'd like to confirm something if I may: can we safely assume that the stride for red, green, and blue planes in planar RGB, and the U and V planes for planar YUV, will always be identical? Or are there circumstances where all the planes could have different pitches?

[Myrsloik]

A plane of the same width and bytesperpixel will always have the same stride.

[sneaker_ger]

You probably want to say "even make it work in VDub"
vfw supports P016

But VS does not support P016 + 4:4:4 VfW output.
Actually I would have preferred to use simple old RGB24 to make some screenshots, but that's not supported yet either.

[sneaker_ger]

"On purpose"? You mean "not yet", right?
Anyways, my goal was to make some screenshots of the 16 bit -> 8 bit dithering of fmtconv and I wanted to quickly use VirtualDub to chooe the frames and copy them to clipboard. My initial post was a bit misleading. Since RGB24 VfW output is not yet supported, 8 bit 4:4:4 YUV (BT709 PC) seemed like the next best choice to me.

[sneaker_ger]

Your post disappearing means you just successfully tested 4:4:4, I guess.

[Chikuzen]

I ported FineSharp (http://forum.doom9.org/showthread.php?p=1569035#post1569035) to VapourSynth.

https://gist.github.com/4111749

This may be an example to port avs scripts.


import vapoursynth as vs
import finesharp
core = vs.Core()
core.avs.LoadPlugin('removegrain.dll')
core.avs.LoadPlugin('repair.dll')
clip = some.source('file')
fs = finesharp.FineSharp(core)
clip = fs.sharpen(clip)


EDIT:
script updated.

[Myrsloik]

I ported FineSharp (http://forum.doom9.org/showthread.php?p=1569035#post1569035) to VapourSynth.

https://gist.github.com/4111749

This may be an example to port avs scripts.


import vapoursynth as vs
import finesharp
core = vs.Core()
clip = some.source('file')
fs = finesharp.FineSharp(core, '/path/to/RemoveGrain.dll', '/path/to/Repair.dll')
clip = fs.sharpen(clip)


Mostly looks ok. I think having filter paths in the script is a bad idea though. Just throw an error if the prerequisite plugins aren't loaded.

You should also be using std.Merge() for your mergeluma function. The goal is to use vs as much as possible, right?

[Chikuzen]

You should also be using std.Merge() for your mergeluma function.

std.Merge(clip[] clips[, int[] weight])
http://www.vapoursynth.com/doc/functions/merge.html

Since I thought that weight could use only int, I used lut2.

[Myrsloik]

std.Merge(clip[] clips[, int[] weight])
http://www.vapoursynth.com/doc/functions/merge.html

Since I thought that weight could use only int, I used lut2.

Documentation typo. It really is float.

[Chikuzen]

- .. function:: Merge(clip[] clips[, float[] weight])
+ .. function:: Merge(clip[] clips, float[] weight=[0.5])

[Revgen]

Since VapourSynth doesn't support audio yet, I've ported/rewritten TheFluff's split_aud script (http://mod16.org/hurfdurf/?p=33) as a Python module for use with VapourSynth.

Name: sam (for "split aud module")
Requirements: mkvtoolnix
Limitations: no non-linear trimming, no vfr support

Typical use case:

video_path=""
audio_path=""
DGDecode=""
VIVTC=""

import vapoursynth as vs, sys, sam
core = vs.Core()
core.avs.LoadPlugin(DGDecode)
core.std.LoadPlugin(VIVTC)
vid = core.avs.MPEG2Source(video_path)
vid = core.vivtc.VFM(vid,1)
vid = core.vivtc.VDecimate(vid)
vid = core.resize.Bicubic(vid,1280,720)
vid = sam.trim(vid,100,200)+sam.trim(vid,13000,13100)
sam.split_merge(audio_path)
last = vid
last.output(sys.stdout, y4m=True)

Source:
http://pastebin.com/q25ea2kN
Testing, thoughts, comments, suggestions and improvements(!) are appreciated.

Thx. I did a quick test on a 1920x1080i 29.97i MPEG-2 file using two trims. Encoded to h264, Muxed the .264 file and the .mka file in MKVToolnix. Audio was in perfect sync. I might try something more complicated next week after Thanksgiving is over.

[Chikuzen]

@HolyWu
I updated that script, and created new thread (http://forum.doom9.org/showthread.php?p=1601585#post1601585).

[Chikuzen]

import vapoursynth as vs
core = vs.Core()
clip1 = core.std.BlankClip(format=vs.YUV420P8, color=[0, 128, 128])
clip2 = core.std.BlankClip(format=vs.YUV420P8, color=[255, 128, 128])
last = core.std.SelectClip([clip1, clip2], clip1, lambda n, f: n % 2)


last.output(file) works as expected.
But VFW module does not.
VirtualDub/AVISource has frozen.

[Myrsloik]

import vapoursynth as vs
core = vs.Core()
clip1 = core.std.BlankClip(format=vs.YUV420P8, color=[0, 128, 128])
clip2 = core.std.BlankClip(format=vs.YUV420P8, color=[255, 128, 128])
last = core.std.SelectClip([clip1, clip2], clip1, lambda n, f: n % 2)


last.output(file) works as expected.
But VFW module does not.
VirtualDub/AVISource has frozen.

I've figured out why. The GIL deadlocks on itself. It's a bit of a mystery why it isn't properly released at the end of script opening and I really can't rule out a cython bug.

Here's a new vsvfw.dll (https://dl.dropbox.com/u/73468194/vsvfw.dll) that should work. I hope you enjoy my ugly hack.

[Revgen]

here's a replacement for bob, at least using avisynth code (i haven't had time to teach myself python yet):

interleave(separatefields().selecteven().bicubicresize(last.width,last.height,1/3.,1/3.,0,.25,last.width,last.height/2),separatefields().selectodd().bicubicresize(last.width,last.height,1/3.,1/3.,0,-.25,last.width,last.height/2))

i think vapoursynth should have enough internal filters to implement the above line, if the resizers are up to it.

It's now possible with fmtconv, but it's pretty slow. Runs at 49-50fps at 1920x1080 on my Q6850 quadcore in vdub. It crawls to 25fps if I use vs.Core(threads=1).

video_path=r'E:\myvideo.d2v'

import vapoursynth as vs
import sys

core = vs.Core()
core.std.LoadPlugin(r'C:\Program Files\VapourSynth\fmtconv.dll')
core.std.LoadPlugin(r'C:\Program Files\VapourSynth\d2vsource_beta3_mingw.dll')
vid = core.d2v.Source(video_path,0)
sep = core.std.SeparateFields(vid,tff=1)
evn = core.std.SelectEvery(sep, cycle=2, offsets=0)
odd = core.std.SelectEvery(sep, cycle=2, offsets=1)
evn = core.fmtc.resample(evn,w=1920,h=1080,sx=0,sy=0.25,sw=1920,sh=540,fulls=0,fulld=0)
odd = core.fmtc.resample(odd,w=1920,h=1080,sx=0,sy=-0.25,sw=1920,sh=540,fulls=0,fulld=0)
evn = core.fmtc.bitdepth(evn,bitdepth=8,fulls=0,fulld=0)
odd = core.fmtc.bitdepth(odd,bitdepth=8,fulls=0,fulld=0)
vid = core.std.Interleave(clips=[evn,odd])
last = vid
#last.output(sys.stdout, y4m=True)

[Chikuzen]

Here's a new vsvfw.dll (https://dl.dropbox.com/u/73468194/vsvfw.dll) that should work. I hope you enjoy my ugly hack.

seems new vsvfw.dll solved that. :thanks:

[Chikuzen]

http://www.vapoursynth.com/doc/pythonbindings.html#raw-access-to-frame-data
"Accessing the data is a bit trickier as get_data() only returns a pointer."

get_data() has already removed because it was deplicate of get_read_ptr()

http://www.vapoursynth.com/doc/functions/planedifference.html
- std.PlaneDifference(clip[] clips, int plane[, string prop = 'PlaneAverage'])
+ std.PlaneDifference(clip[] clips, int plane[, string prop = 'PlaneDifference'])

[osgZach]

GPU stuff is overhyped. Proper coding and common sense will get you almost as far for most filters. I'll probably add support for it in some form one day anyway just to prove that I'm right.
A few facts about python. It's really slow and in practice single threaded. The c libraries behind the calculation modules will run much faster if used directly in a c plugin.
Maybe it could be usable for filter prototyping for some people but I predict a huge yawn there. Video filters also aren't really "scientific calculations", they're more of an exercise in stuffing as many pixels into an xmm register as possible.

I don't know... Sure todays CPU's are pretty powerful, and a good multi-threaded app would be able to do a lot.. But if its one thing video filtering with Avisynth has taught me, it is that when you need to do a lot of heavy filtering on a source, it can bring even a beefy CPU to its knees. I can only imagine what such work on an HD source might be like, but still, I also am reminded that a lot of users here are using not necessarily top-end/modern gear.

So it seems to me any opportunity to dump additional workload off onto other systems, such as a GPU, is still something worth investigating in many circumstances. Especially if that means a given job might go at 5 or 10 FPS, vs 1 FPS (depending on what you are doing).

Sure, GPU accelerated encoding has kind of turned out to be marketing hype / a bit of a flop. But I still would like to think there is potential to be uncovered in GPU assisted filtering and processing of video.

Should it be a high development priority? No, I wouldn't be stupid enough to suggest such a thing. But I don't think it should be overlooked or so easily written off, on the other hand. Of course I could totally be wrong. I don't pretend to be a professional, or extremely knowledgeable about programming (especially as it pertains to video processing), but I guess that's just my simple opinion of things.

[Myrsloik]

Here's something new, a code doodle just to see what people think. It works a lot like mt_lut(xy(z)) but instead of using a lut it directly computes the expression. It has most of the same operators too with the most notable exception that ^ is written pow.

I simply call it Expr (https://dl.dropbox.com/u/73468194/expr.dll).
Usage:
core.expr.Expr([clipa, clipb], expr="x y +", plane=0)

You supply 1-3 input clips, these can then be referred to with x, y and z in the expression. It is allowed to mix and match the input clips quite a bit, the only requirement is that the plane that's being evaluated has the same width and height in all the inputs. The output format and dimensions are taken from the first clip.

All computations are done in float precision and the loaded values are not normalized in any way when loaded. So an 8 bit input will be in the 0-255 range, a float clip will be 0-1 or -0.5 to +0.5 and a 10 bit clip would be 0-1023.

NOTE THAT IT HAS ALMOST NO ERROR CHECKING SO A TYPO WILL GIVE UNPREDICTABLE RESULTS OR A CRASH
I DON'T WANT ANY REPORTS ABOUT THOSE PROBLEMS JUST OPINIONS ON THE IDEA AND EXECUTION

Operator list:
+ - * / max min
exp log pow sqrt abs
> < == ?
sin asin cos acos tan atan tanh

If people here like the structure and idea I'll go and write the whole thing in asm which should speed it up by quite a bit. Maybe a version that evaluates expressions in integer form would be interesting too? I don't know. Poke it a bit and say what you think about the idea.

[Myrsloik]

Can the vapoursynth.VideoNode.output() method write to an IO byte stream object like io.BytesIO()?

No. Only things with real file handles allowed. (Has a fileno method)
It kinda has to do with threading.

[Mug Funky]

Expr seems to (so far in my 10 mins of testing) only work as expected on YUV444P8.

P16 gives me a half-width picture with junk filling the rest of the space.

also, output seems restricted to 0-255 range on 10 or 16 bit input. from your post i would have expected 0-1023 for 10-bit and 0-65535 for 16-bit. exceeding 255 leads to wrap-around integer fun.

[edit]

not being ungrateful of course. it runs very fast and is pretty elegant. i expect bells and whistles to be included when you've decided whether to continue with it. it's good to have more tools in the ol' arsenal :). and processing per-pixel rather than building a big lut mans more CPU gives faster results rather than more memory. this can be a definite advantage :)

[Chikuzen]

How do I set the planes argument of VS's eedi3 to achieve the same result of original eedi3(Y=true, U=false, V=false) or eedi3(Y=false, U=true, V=true)? TIA.

eedi3(Y=true, U=false, V=false)
-> core.eedi3.EEDI3(clip, planes=0)

eedi3(Y=false, U=true, V=true)
-> core.eedi3.EEDI3(clip, planes=[1, 2])

[Eyldebrandt]

So, just a question.
Can we use vapoursynth in real time for post-processing, as we do with avisynth ?
How can we do, if appropriate ?

[ryrynz]

Can we use vapoursynth in real time for post-processing, as we do with avisynth ?


See below.

Are there any plans for a DirectShow filter that can serve as a replacement for ffdshow's AviSynth processing?


Not at this time. It'd be pretty easy to implement for someone who is familiar with dshow though.

[Mug Funky]

i'm having some trouble getting VS to compile on my msys environment.

it's stopping at "Checking for header QtCore/QtCore". i've dl'd, compiled and installed Qt opensource, but i've no idea where it installed to and whether i even compiled the right modules. it's a huge codebase and there's no configure option to just compile QtCore. i've also no idea where it decided to install to...

is there a trick to making qtcore visible to waf? most things i've installed use pkgconfig to find everything, but python and waf and qt are all quite new to me, so i can't make that assumption.

thanks!

[edit]

actually, if i knew exactly what qt tools the waf configure command was looking for, i could place them in mingw/bin or /lib, /include etc.

[Myrsloik]

i'm having some trouble getting VS to compile on my msys environment.

it's stopping at "Checking for header QtCore/QtCore". i've dl'd, compiled and installed Qt opensource, but i've no idea where it installed to and whether i even compiled the right modules. it's a huge codebase and there's no configure option to just compile QtCore. i've also no idea where it decided to install to...

is there a trick to making qtcore visible to waf? most things i've installed use pkgconfig to find everything, but python and waf and qt are all quite new to me, so i can't make that assumption.

thanks!

[edit]

actually, if i knew exactly what qt tools the waf configure command was looking for, i could place them in mingw/bin or /lib, /include etc.

It's not supported under windows. I see no reason for there to be inferior (no avisynth compatibility) binaries floating around.

[Myrsloik]

I made a new version of my Expr (https://dl.dropbox.com/u/73468194/expr.dll)filter. I'll probably add it to the core once it's stable since it's a good alternative to using a lut for higher bitdepths.

Here's a slightly modified recap from the previous post.

usage:
core.expr.Expr([clipa, clipb, clipc], expr=["x y + z +", ""], plane=0)

You supply 1-3 input clips, these can then be referred to with x, y and z in the expression. It is allowed to mix and match the input clips quite a bit, the only requirement is that the plane that's being evaluated has the same width and height in all the inputs. The output format and dimensions are taken from the first clip.

By default the expression is applied to all planes, supply an array to apply different expressions. The plane of the first clip (clipa) will be copid if an empty expression is given and the output format is the same as the format of clipa. (this can be changed with format=vs.YUV444P8 or any other format constant). In the cases the plane cannot be copied due to format mismatch the output will be undefined.

All computations are done in float precision and the loaded values are not normalized in any way when loaded. So an 8 bit input will be in the 0-255 range, a float clip will be 0-1 or -0.5 to +0.5 (Y and UV respectively) and a 10 bit clip would be 0-1023.

Operator list:
+ - * / max min
sqrt abs
Comparison: > < = ?
Comparisons return 1.0 for true and 0.0 for false. The ternary operator ? consider all values > 0.0 as true.
Logical: and or xor not
Logical operators consider all values > 0.0 as true and return 1.0 for true and 0.0 for false.

Missing but will probably be added some day: exp log pow

It is faster than mt_lutxy for very simple expressions such as "x y +" and equal for "x y - 128 +". Overall speed of course depends on both expression length and instruction complexity but is still very fast even on a complicated expression.

[Are_]

Ok, I gave it a try, and it worked fine for 8 bit clips, but failed for anything else.

This produced a clip with white horizontal stripes (and pink ones if applied to chroma):

src = core.ffms2.Source(source=r'D:\ENCODE\vs\vid.mkv')
res = src
res = core.fmtc.bitdepth(res, vs.YUV420P10)
res = core.expr.Expr(res, ['x 100 +','',''])
last = core.fmtc.bitdepth(res, vs.YUV420P8)

Changing the two last lines to:
res = core.expr.Expr(res, ['x 100 +','',''], format=vs.YUV420P8)
last = res

Produced a solid white/pink clip, no stripes (you can still see the picture, but hardly).

Am I doing anything wrong? :/

EDIT:Oh, and no plane argument:
VapourSynth Expr Filter
namespace: expr
identifier: com.vapoursynth.expr
Expr(clips:clip[]; expr:data[]; format:int:opt;)

[Myrsloik]

Ok, I gave it a try, and it worked fine for 8 bit clips, but failed for anything else.

This produced a clip with white horizontal stripes (and pink ones if applied to chroma):

src = core.ffms2.Source(source=r'D:\ENCODE\vs\vid.mkv')
res = src
res = core.fmtc.bitdepth(res, vs.YUV420P10)
res = core.expr.Expr(res, ['x 100 +','',''])
last = core.fmtc.bitdepth(res, vs.YUV420P8)

Changing the two last lines to:
res = core.expr.Expr(res, ['x 100 +','',''], format=vs.YUV420P8)
last = res


Produced a solid white/pink clip, no stripes (you can still see the picture, but hardly).

Am I doing anything wrong? :/

EDIT:Oh, and no plane argument:
VapourSynth Expr Filter
namespace: expr
identifier: com.vapoursynth.expr
Expr(clips:clip[]; expr:data[]; format:int:opt;)

Maybe. Remember that it's not normalized based on the input so a 10 bit clip with 8 bit output will be like x 4 * 100 + in your expression. Divide
X by 4 to get something equivalent. There could also be a bug in the 9-16 bit code

[ajp_anton]

Operator list:
+ - * / max min
sqrt abs
Comparison: > < = ?
Comparisons return 1.0 for true and 0.0 for false. The ternary operator ? consider all values > 0.0 as true.
Logical: and or xor not
Logical operators consider all values > 0.0 as true and return 1.0 for true and 0.0 for false.

Missing but will probably be added some day: exp log powpow should be named ^.
What about !=, >= and <=? Instead of combining =,<,> with a not.

[Myrsloik]

pow should be named ^.
What about !=, >= and <=? Instead of combining =,<,> with a not.

I suppose I should add the additional shades of equality. Just run off and test it and maybe I'll actually implement pow some day...

Since this is a fairly simple implementation I may add some stack manipulation operators too just to see what happens- I predict confused users.

[ajp_anton]

My programming knowledge is non-existant, but if I understood correctly from wikipedia what stack manipulation (or even a stack) is, then it's something I very much welcome =). At least what that one wiki page calls "dup" and "swap" would be useful.

[jmac698]

Why stop there? Write a Forth interpreter ;)

[fvisagie]

Apologies if this has been discussed, in which case I missed it. My question is: could VapourSynth be the way forward for Avisynth?

To me as an Avisynth user with limited understanding, it comprises the following main components:
1. frame server
2. rich built-in and external functions (filters)
3. rich programmability for user functions (parser)

Each of which represents a vast investment in development time and vested interests in terms of uers' tool chain and workflow.

There seem to issues holding up Avisynth development progress; specifically getting multi-threading to work correctly as I understand it. To me that would be a frame server issue, meaning nothing intrinsically wrong with its ecosystem of filters and functions - only the frame server.

On the other hand, a frame server technology refresh seems to be exactly one of the main goals of VapourSynth. Along with that comes the possibility to support (with extra development) Avisynth filters as well as its parser. It even seems possible for this support to come in various forms, e.g. creating an Avisynth emulation layer (most functionality available immediately but possibly with (future) restrictions) vs. porting individual filters and the parser to the VapourSynth Python environment (possibly longer time-to-market and also relying to some extent upon individual filter authors to contribute).

However, whichever way one looks at it, it seems technically *possible* for a future Avisynth based on the VapourSynth frame server to look, feel and work just like the current one, except without the issues plagueing the current one hopefully.

And, if it were possible to unify the two projects, everyone, absolutely everyone involved stands to benefit. On the face of it, that seems a very attractive prospect.

Therefore, to rephrase the question, has an analysis been done comparing the feasability of continuing with the Avisynth frame server vs. porting everything but the problematic part/s to VapourSynth, and what were the results and outcome? If nothing of the sort, what would be the merit of looking into it?

Thanks for your time,
Francois

[Myrsloik]

VaporSynth has a very clearly defined core library. You could fairly easily add another scripting front end. I actually considered adapting the script parser from avisynth but as a general language it also has a few issues too many (no arrays, ifs and butts).

Getting threading to work properly in avisynth has a lot of issues. The main one (from my understanding) being that the PClip and PVideoFrame smart pointer types aren't thread safe. They're also compiled into every plugin so breaking binary compatibility and recompiling all plugins is a must unless you're also willing to restructure the avisynth internals completely. For example by making each filter execute in its own virtual avisynth environment to keep binary compatibility and get reasonable multithreaded properties. (coincidentally what vs does)

Since a compatibility layer will be needed, we may as well modify the core part a bit too, right? Add those nice missing features such as arrays, a better C api, portability. Oh wait, the avisynth core is a mess where everything is tangled so let's separate the core from the scripting first. (coincidentally this is vapoursynth.dll)

While we're busy and binary compatibility is no longer an issue why not learn from the mistakes of avisynth 3? Writing a parser and scripting language is a lot of work and can kill the project. So let's adapt an existing one, like python or lua. (coincidentally what vs does)

So basically I see VapourSynth as the logical next step. If you go back and fix enough things this is what you'll get. As for uniting the projects there really isn't that much more that should be shared in my opinion, the only exception being ideas for filters and scripts.

The outcome: this?

[fvisagie]

Edit: I should have added: fair comment.

There is not an iota of truth in this.

That statement's already been idemnified ("To me as an Avisynth user with limited understanding") ;).

But seriously now, in my view Avisynth's wide appeal is due in large part to the ease and simplicity with which its power and flexibility can be accessed - in other words, to its accessibility. It does not need "programming". Avisynth arguably struck as close to the ideal balance between power and accessibility as can be done.

On the other hand, people who write Python code are seen as "programmers". So, while I do not claim to know in which direction Vapoursynth is headed, one thing seems fairly certain, and that is if it doesn't support a scripting interface of similar simplicity as Avisynth's, it won't have as wide an appeal.

So, while creating a technically superior framework no doubt is rewarding in its own right, in my opinion it would be a shame if it were to be self-limiting for lack of accessibility.

[sneaker_ger]

To be fair, creating simple vs scripts is not that much more difficult than creating simple AviSynth scripts. Working with high bit depth stuff is even easier. The biggest downside of vs (aside from lacking many plugins and scripts that AviSynth has) is the missing audio support.

The few people that can actually write powerful AviSynth scripts will have no problem figuring out Python.

[Revgen]

On the other hand, people who write Python code are seen as "programmers". So, while I do not claim to know in which direction Vapoursynth is headed, one thing seems fairly certain, and that is if it doesn't support a scripting interface of similar simplicity as Avisynth's, it won't have as wide an appeal.


I'm not a programmer and I don't find VS anymore difficult to use than avisynth. They are actually pretty similar. The only real downside is that VS's documentation isn't as robust.

[fvisagie]

The few people that can actually write powerful AviSynth scripts will have no problem figuring out Python.

No arguing with that; it's the other users where my concern lies.

But thanks for your response, valid as it is - I'm not here to argue, I simply wanted to make a point about user appeal for the Vapoursynth community to consider. Or not ;).

[JEEB]

Let's not forget that Python is just the language that was picked for the first Vapoursynth scripting mappings.

Anyone is free to add vapoursynth mappings for the language of their choice. The fact that the library itself is just something with a C API lets you do that.

[Myrsloik]

I updated Expr again.

Added operators: exp log (as in natural logarithm) <= >=

The only thing I think is really missing now is pow. Tanh could also be useful in some cases but apart from that consider it complete but somewhat untested.

If someone can give me an example where stack manipulation would speed up/simplify an expression a lot I'll add it. (such as duplicate top element on stack and swap top with second to top element)

Note that a clever person can now use Expr to do gamma aware resizing with fmtconv.

Download it (https://dl.dropbox.com/u/73468194/expr.dll)

[ajp_anton]

Say you have an expression for f(x,y,z) that is rather complicated so you don't want to repeat it. You need g(f) + h(f), where g and h are some simple operators.
"f dup g swap h +"
which I hope means it will duplicate f, apply g on one of them, swap them so you can use h on the other one, and them add them.

I don't really have a real-world example right now, but I've stumbled upon this scenario in the past.

[lansing]

any news on adding the load virtualdub plugin function yet?

[Mug Funky]

Note that a clever person can now use Expr to do gamma aware resizing with fmtconv.

hopefully i am clever enough.

thanks for the log and exp though - my plan for the moment was not so much gamma-aware but log aware (ie film land) resizing and whatnot.

this way one can manipulate 10-bit film scans without compromise.

now... if i can just get temporalsoften to play nice :)

[edit]

there still seem to be problems with 9-16 bits. looks like strips of 4 good pixels with YUV black in between.

[Gavino]

Say you have an expression for f(x,y,z) that is rather complicated so you don't want to repeat it. You need g(f) + h(f), where g and h are some simple operators.
"f dup g swap h +"
A simple way to do things like that is to save the f expression as a separate string and include it in the main expression where required.

f = " ... " # long string including x, y, z, etc
expr = f+" g "+f+" h +"

Of course, the f expression then gets calculated twice, but at least you don't have to copy it yourself and can more easily see if you have written the final expression correctly.

[ajp_anton]

Of course, the f expression then gets calculated twiceYes, that was the point, to make it faster. With mt_lut it didn't matter much, so I just made a string I could use multiple times, but that's not the case here.

[Reel.Deel]

It's a bug in mvtools. For some reason it doesn't use the emms instruction to restore the fpu state before returning when pel=1. Contact the mvtools author to get it fixed. I will never add a workaround for this in vs because it really is a fatal error.

MVTools SVP edition (2.5.11.9) has been updated (http://www.svp-team.com/forum/viewtopic.php?id=1069) to try to correct the bug with pel=1.

[Chikuzen]

I noticed a weird behavior of VFW.
Opened task manager after VirutalDub closed, the process was still on there and not gone without killing it manually.
Is this known issue ?

[Myrsloik]

I noticed that a weird behavior of VFW.
Opened task manager after VirutalDub closed, the process was still on there and not gone without killing it manually.
Is this known issue ?

Can't reproduce. It always happens?

[Chikuzen]

hmm, It always happens on my box.
and VirtualDub always crashes when I hit F2(Reopen video file).

[Myrsloik]

hmm, It always happen on my box.
and VirtualDub always crash when I hit F2(Reopen video file).

That's because the vsvfw.dll I posted is a bit buggy. I have improved the hack.

Here's the latest test build. (https://dl.dropbox.com/u/73468194/vsvfw.dll)

Debugging AVISource now but progress is slow...

[Keiyakusha]

hmm, It always happen on my box.
and VirtualDub always crash when I hit F2(Reopen video file).
Haven't noticed that the process stays there, need to check, but it crashes for me too.

[Chikuzen]

It seems that latest vsvfw.dll solved that.

[Keiyakusha]

Still crashes for me. But before it was each time I press F2 and crash was in msvcr100 (or something like that), now it crashes after I press F2 few times (3-4) and the crash in python33.dll

[Myrsloik]

Still crashes for me. But before it was each time I press F2 and crash was in msvcr100 (or something like that), now it crashes after I press F2 few times (3-4) and the crash in python33.dll

Never mind, reproduced. There's a big memory leak now instead.

[sneaker_ger]

You mean the one I reported on November 19th? :rolleyes:

[Myrsloik]

You mean the one I reported on November 19th? :rolleyes:

I thought it was only a problem related to the other deadlocks. I'll try to fix it quickly and then release a new version...

[pbristow]

Chikuzen:
Opened task manager after VirutalDub closed, the process was still on there and not gone without killing it manually.

A possibly relevant observation: Since moving to Win7 I see lots more instances of such zombie processes, that have to be killed manually, than I used to on WinXP (not just with VirtualDub, but with IE, Firefox, various games, and so on.) It may be due to change in the way Windows is handling certain error conditions...(?)

[Myrsloik]

A possibly relevent observation: Since moving to Win7 I see lots more instances of such zombie processes, that have to be killed manually, than I used to on WinXP (not just with VirtualDub, but with IE, Firefox, various games, and so on.) It may be due to change in the way Windows is handling certain error conditions...(?)

1. I use windows 7 too.
2. There's no shortage of actual bugs in python part. I hate cython now.

I've so far found half the reason for the leak. A reference leak in the core I added by accident together with the multiple output support. The other leak issue (bug on top of bug) I'm still huting...

[qyot27]

The new eedi3 failed to build when I tried yesterday on 64-bit Linux. Errored out with something about C99 and for loops.

Also, not sure what's going on here, but:
Source video is high bit depth H.264 (files were 12-bit and 14-bit, 4:2:0, 4:2:2, and 4:4:4)
Script:
#!/usr/bin/env python3
import vapoursynth as vs
import sys
core = vs.Core()
core.std.LoadPlugin('/usr/local/lib/libffms2.so')
ret = core.ffms2.Source("12bit-420.mkv")
ret.output(sys.stdout, y4m=True)
The following fails (complains that it can't recognize file format):
./testscript.vpy | mpv -
I even tried going down to 10-bit in-script, just in case there was some weirdness concerning the file's bit depth, but it still couldn't do it.

This works though:
./testscript.vpy | ffmpeg -i - -f yuv4mpegpipe -strict -1 - | mpv -
And mpv correctly recognizes the y4m stream as yuv***p16le.

However, testing on Windows, the official build of R14 worked without me having to pipe through ffmpeg first.

[Are_]

for loops are c99, default mode for gcc is a flavour of c90 because c99 features are incomplete.

To fix your problems with mplayer you should specify the demuxer like this:
$ python3 testscript.vpy | mplayer2 -demuxer y4m -

I guess it's the same for mpv.

[Myrsloik]

The eedi3 error is because I committed jackoneill's conversion to c without testing it myself. Just revert that dir one commit and it will work. I'll make it proper old school c this evening.

[qyot27]

To fix your problems with mplayer you should specify the demuxer like this:
$ python3 testscript.vpy | mplayer2 -demuxer y4m -

I guess it's the same for mpv.
It's not. MPV has no discrete yuv4mpeg demuxer. It uses libavformat's.

But that's beside the point, because I figured out what the issue was after seeing it fail when I built R14 for Linux. I needed to update Python. The repo version pulled in by the python3 package is 3.2.3. After I installed 3.3 it started working as it should. 3.3 was the first/only version I have of the 3.x series on Windows, so that's why it was okay on there.

[Yellow_]

hi, currently trying to build vapoursynth on 64bit 12.10 Ubuntu Linux, done the .waf configure, think I've met all dependancies, failing on build at:

[12/17] c: src/filters/eedi3/eedi3.c -> build/src/filters/eedi3/eedi3.c.3.o
../src/filters/eedi3/eedi3.c: In function ‘interpLineFP’:
../src/filters/eedi3/eedi3.c:77:7: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c:77:7: note: use option -std=c99 or -std=gnu99 to compile your code
../src/filters/eedi3/eedi3.c:80:10: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c:83:13: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c:96:7: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c:99:10: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c:102:13: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c:110:16: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c:119:16: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c:135:4: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c:142:7: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c:147:10: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c:164:13: error: redefinition of ‘x’
../src/filters/eedi3/eedi3.c:135:13: note: previous definition of ‘x’ was here
../src/filters/eedi3/eedi3.c:164:4: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c:167:13: error: redefinition of ‘x’
../src/filters/eedi3/eedi3.c:164:13: note: previous definition of ‘x’ was here
../src/filters/eedi3/eedi3.c:167:4: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c: In function ‘interpLineHP’:
../src/filters/eedi3/eedi3.c:200:4: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c:220:7: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c:223:10: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c:229:16: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c:238:16: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c:252:7: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c:255:10: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c:261:16: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c:270:16: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c:280:16: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c:289:16: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c:304:13: error: redefinition of ‘x’
../src/filters/eedi3/eedi3.c:200:13: note: previous definition of ‘x’ was here
../src/filters/eedi3/eedi3.c:304:4: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c:311:7: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c:316:10: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c:333:13: error: redefinition of ‘x’
../src/filters/eedi3/eedi3.c:304:13: note: previous definition of ‘x’ was here
../src/filters/eedi3/eedi3.c:333:4: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c:336:13: error: redefinition of ‘x’
../src/filters/eedi3/eedi3.c:333:13: note: previous definition of ‘x’ was here
../src/filters/eedi3/eedi3.c:336:4: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c: In function ‘copyPad’:
../src/filters/eedi3/eedi3.c:381:7: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c:391:7: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c:399:4: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c:407:7: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c:409:10: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c:412:19: error: redefinition of ‘x’
../src/filters/eedi3/eedi3.c:409:19: note: previous definition of ‘x’ was here
../src/filters/eedi3/eedi3.c:412:10: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c:417:16: error: redefinition of ‘y’
../src/filters/eedi3/eedi3.c:407:16: note: previous definition of ‘y’ was here
../src/filters/eedi3/eedi3.c:417:7: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c:421:16: error: redefinition of ‘y’
../src/filters/eedi3/eedi3.c:417:16: note: previous definition of ‘y’ was here
../src/filters/eedi3/eedi3.c:421:7: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c: In function ‘eedi3GetFrame’:
../src/filters/eedi3/eedi3.c:468:7: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c:485:10: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c:507:13: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c:518:19: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c: In function ‘eedi3Create’:
../src/filters/eedi3/eedi3.c:679:7: error: ‘for’ loop initial declarations are only allowed in C99 mode
../src/filters/eedi3/eedi3.c: In function ‘eedi3GetFrame’:
../src/filters/eedi3/eedi3.c:463:7: warning: ignoring return value of ‘posix_memalign’, declared with attribute warn_unused_result [-Wunused-result]
../src/filters/eedi3/eedi3.c:465:7: warning: ignoring return value of ‘posix_memalign’, declared with attribute warn_unused_result [-Wunused-result]
Waf: Leaving directory `/home/xeon/Apps/vpsynth/build'
Build failed
-> task in 'eedi3' failed (exit status 1):
{task 17620048: c eedi3.c -> eedi3.c.3.o}
['/usr/bin/gcc', '-DVSCORE_EXPORTS', '-fPIC', '-O3', '-I/home/xeon/Apps/vpsynth/build/include', '-I/home/xeon/Apps/vpsynth/include', '-DHAVE_QTCORE=1', '-DHAVE_QTGUI=1', '-DHAVE_QTUITOOLS=1', '-DHAVE_QTNETWORK=1', '-DHAVE_QTOPENGL=1', '-DHAVE_QTSQL=1', '-DHAVE_QTSVG=1', '-DHAVE_QTTEST=1', '-DHAVE_QTXML=1', '-DHAVE_QTXMLPATTERNS=1', '-DHAVE_QTWEBKIT=1', '-DHAVE_QT3SUPPORT=1', '-DHAVE_QTHELP=1', '-DHAVE_QTSCRIPT=1', '-DHAVE_QTDECLARATIVE=1', '-DFEATURE_SHARED=1', '-DFEATURE_STATIC=0', '-DFEATURE_FILTERS=1', '-DFEATURE_CYTHON=1', '-DFEATURE_AVISYNTH=1', '-DFEATURE_DOCS=0', '-DFEATURE_EXAMPLES=0', '-DPATH_PREFIX="/usr/local"', '-DPATH_LIBDIR="/usr/local/lib"', '-DPATH_PLUGINDIR="/usr/local/lib/vapoursynth"', '-DPATH_DOCDIR="/usr/local/share/doc/vapoursynth"', '-DPATH_INCLUDEDIR="/usr/local/include/vapoursynth"', '-DHAVE_QTCORE_QTCORE=1', '-DHAVE_QATOMICINT=1', '-DHAVE_LIBAVUTIL_AVUTIL_H=1', '-DHAVE_AVUTIL_LICENSE=1', '-DHAVE_LIBSWSCALE_SWSCALE_H=1', '-DHAVE_SWSCALE_LICENSE=1', '../src/filters/eedi3/eedi3.c', '-c', '-o', 'src/filters/eedi3/eedi3.c.3.o']


Cheers

[Youka]

Isn't the error message very clear?
You're using gcc to compile eedi3.c as C source code. Old C doesn't allow declarations inside a for-loop header, like it's the case in this file, so you have to add C99 mode to compiler options for modern C to allow it.

[Yellow_]

Youka, thanks for the reply, I'm no coder so it isn't clear at all to me, I'm having to compile because I can find no PPA for Vapoursynth, so could you elaborate on how I add C99 mode? I google I find std=c99 but how do I set that for using Vapoursynths ./waf build ?

[Adub]

Actually, if you look up a few posts, you'll see Myrsloik's answer to this very problem.
linky (http://forum.doom9.org/showthread.php?p=1605209#post1605209)

[Yellow_]

Adub & Youka, thanks, so the 'answer' to my query was in fact:

svn checkout -r 365 http://vapoursynth.googlecode.com/svn/trunk/ vapoursynth-365

Or wait for a rectified commit.

-r 365 did the job compiling, now I run ./waf test and error:

ImportError: No module named vapoursynth

whereis vapoursynth, answer:

vapoursynth: /usr/local/lib/vapoursynth

whereis vapoursynth.so:

vapoursynth: /usr/local/lib/vapoursynth /usr/local/include/vapoursynth

libvapoursynth.so exisits in /usr/local/lib

[qyot27]

You have to run 'sudo ./setup.py install' after 'sudo ./waf install' to get the Cython module built and installed. Otherwise you just have the library and nothing that links it into Python.

Also note what I mentioned in my last post about the need for using Python 3.3 if you want yuv4mpeg pipes to work like they should. Ubuntu 11.10-12.10 ship with 3.2 if you grab the python3 package from the repositories (this finally changes with 13.04, which uses 3.3 for its python3 package). What you need are the python3.3 and python3.3-dev packages as well. For good measure you also probably want to compile Cython 0.17.1 or 0.17.2, although I'm not quite as certain on this one (again, 13.04 won't be affected by this; it has 0.17.2).

[Adub]

You're going to need to install Cython via pip in order to get the correct version, as the Ubuntu repositories have a much older version.

"pip install cython" should do it.

[sl1pkn07]

when build vapoursynth in archlinux (https://aur.archlinux.org/packages/vapoursynth-svn/) via makepkg

namcap (package checker) says:

sl1pkn07@sL1pKn07 vapoursynth-svn $ namcap vapoursynth-svn-383-1-x86_64.pkg.tar.xz
vapoursynth-svn W: ELF file ('usr/lib/libvapoursynth.so') has text relocations.
vapoursynth-svn W: ELF file ('usr/lib/libvapoursynth.so') has executable stack.

execstack: https://bugs.archlinux.org/task/26458
text relocation: https://bugs.archlinux.org/task/26434

my linker flags (makepkg.conf)

CFLAGS="-march=x86-64 -mtune=native -O2 -pipe -fstack-protector --param=ssp-buffer-size=4 -D_FORTIFY_SOURCE=2"
CXXFLAGS="-march=x86-64 -mtune=native -O2 -pipe -fstack-protector --param=ssp-buffer-size=4 -D_FORTIFY_SOURCE=2"

executable stack fix with: "execstack -c build/libvapoursynth.so" (adding "-Wa,--noexecstack" in CFLAGS don't work)
i don'k know fix text relocation issue

greetings

[Are_]

executable stack fix with: "execstack -c build/libvapoursynth.so" (adding "-Wa,--noexecstack" in CFLAGS don't work)

Adding -z,noexecstack to the LDFLAGS fixed it for me, I didn't try with the CFLAGS. (I'm on a different distro, but I'm almost sure your makepkg.conf is ignored, you have to manually modify wscript for custom C/LDFLAGS).

[sl1pkn07]

tnx man, work! without edit wscript, only and LDFLAGS+=",-z,noexecstack"

like this:


LDFLAGS+=",-z,noexecstack"
waf configure --prefix="/usr" --docs="true"
waf build $MAKEFLAGS
python setup.py build


but i think need "fix" in upstream

and no, makepkg.conf not ignore

greetings

[qyot27]

You're going to need to install Cython via pip in order to get the correct version, as the Ubuntu repositories have a much older version.

"pip install cython" should do it.
I also noticed that pipes stopped working right after installing python3-pip, so I'm not sure what's going on. It could be that what I'm seeing isn't a problem with pip (rather, it would be the python3-setuptools that it relies on), but with something else, like I describe below.

If there is something the matter with it there, installing Cython manually is really the only choice. It's just one line - sudo python3.3 setup.py install - after unpacking the tarball.



It seems that if I pipe from VS to MPV, it only works the first time. I usually exit MPV before playback stops of its own accord, since I'm really only testing if the pipe worked. Subsequent times it seems to fail, even with 3.3 installed (although again, on Windows it doesn't do this and pipes correctly every time, making me think there's something wrong with my python setup on Ubuntu - which is problematic, because aside from Cython and VapourSynth, it's repository-standard).

[Myrsloik]

If you haven't noticed it already all the compilation issues should be fixed now. The next release has a big pile of fixes and is coming soon but I've got one odd bug left to fix in vsfs before its ready. The vfw memory leak bug when reopening will most likely not get fixed in R17 because it turned out to be a very tricky one.

[sl1pkn07]

https://code.google.com/p/vapoursynth/source/detail?r=385

don't make ELF without execstack

[Myrsloik]

https://code.google.com/p/vapoursynth/source/detail?r=385

don't make ELF without execstack

I don't dabble much in the non-windows builds but a motivation would be nice.

[jackoneill]

https://code.google.com/p/vapoursynth/source/detail?r=385

don't make ELF without execstack
It's fixed in r388.

[sl1pkn07]

zankius

[Tima]

I have FFV1 avi file which hangs during processing when using stdout to x264, when being opened by avisource.AVISource(), but works fine with ffms2.Source() (all modes).

test.cmd:
python test.vpy | x264_64.exe --demuxer y4m --output "test.mp4" -
test.vpy:
import vapoursynth as vs
import sys
core = vs.Core()

core.std.LoadPlugin(r'C:\Bin\VapourSynth\filters\ffms2.dll')
core.std.LoadPlugin(r'C:\Bin\VapourSynth\filters\avisource.dll')

#clip = core.ffms2.Source('test.avi')
clip = core.avisource.AVISource('test.avi')

#last=clip

clip.output(sys.stdout, y4m=True)


test.avi:
http://rghost.net/42392962

[Chikuzen]

＠Myrsoik
How about adding _SceneChange to reserved frame property ?
If a source filter or an independent scene change detect filter set it to the clip, almost of all temporal filters will use it.
I think it is reasonable.

[Myrsloik]

＠Myrsoik
How about adding _SceneChange to reserved frame property ?
If a source filter or an independent scene change detect filter set it to the clip, almost of all temporal filters will use it.
I think it is reasonable.

Added to the documentation

[Myrsloik]

Here's a beta for you to try. The transpose filter is broken but otherwise it should work better (or equally bad) as r16 at least. Plenty of bug fixes and I just didn't have time to test it properly before my Christmas trip. Coding will continue the 27th so report any other issues you find that aren't on the bug tracker already.

VapourSynth r17 beta (https://dl.dropbox.com/u/73468194/vapoursynth_installer_r17_beta.exe)

[Revgen]

Thank you.

Have a Merry Christmas and enjoy your trip.

[sneaker_ger]

The installer complains about missing Python if Python was installed for the current user only.

Also, is installing a new version on top of the old one OK? I saw that the de-installer deletes some files after the restart, so you would have to restart at least once if you want to upgrade.

[Myrsloik]

The installer complains about missing Python if Python was installed for the current user only.

Also, is installing a new version on top of the old one OK? I saw that the de-installer deletes some files after the restart, so you would have to restart at least once if you want to upgrade.

That's only if the files are in use. As in you've got a script open while upgrading and that is very unlikely to work properly. The installer also simply runs the uninstaller first to try to clear out old stuff and only overwrites after that. I believe it should always work properly and won't bother looking at it unless you can actually make it break.

I'll try to fix the python detection at least for r17.

[Tima]

Vapoursynth doesn't work with latest stable ffms2 v2.17.

Latest beta builds from googlecode work, but index most of my *.avi (Lagarith+AC3) files incorrectly, and don't accept index files from v2.17.

[Myrsloik]

Vapoursynth doesn't work with latest stable ffms2 v2.17.

Latest beta builds from googlecode work, but index most of my *.avi (Lagarith+AC3) files incorrectly, and don't accept index files from v2.17.

2.17 is ancient and not really supported anymore. 2.18 would've been released if ffmpeg/libav weren't so busy breaking their own api.

Also, you'll have to elaborate on "doesn't work". And what is incorrect indexing to you anyway?

[StainlessS]

If ffms2 v2.17 is not recommended, what is, r722 (24K+ downloads) or r725 (2K+ downloads) ?
Thank you in advance.

[Myrsloik]

If ffms2 v2.17 is not recommended, what is, r722 (24K+ downloads) or r725 (2K+ downloads) ?
Thank you in advance.

It is what it says. The svn revisions r722 and r275 which are veeeeery different. 2.17 doesn't even compile with new ffmpeg/libav.

[StainlessS]

Scratches head, but no wiser. :confused:
Thank you anyway.

[Tima]

Also, you'll have to elaborate on "doesn't work".

I get this error when trying to open *.vpy script with ffms2.dll v2.17 in VirtualDub:
---------------------------
VirtualDub Error
---------------------------
Avisynth open failure:
Python exception: 'No entry point found in C:\\Bin\\VapourSynth\\filters\\ffms2.dll'
---------------------------
OK
---------------------------


With r722 I get error message about deprecated R1 API.

And what is incorrect indexing to you anyway?

With r725, ffmsindex.exe instantly makes 124-byte long index for one 2.2Gb file, and I get a crash in Vdub when opening corresponding *.vpy:

An out-of-bounds memory access (access violation) occurred in module 'ffms2'...
...reading address 00000004.

My source is interlaced FFV1+AC3 *.avi file, captured by vdub.
Seems it has something to do with file length, because after trimming and direct stream copy to another (smaller) file, it indexes fine.

Update:
It definitely has something to do with 2Gb file size boundary.
I stripped audio from my source and trimmed down remaining video several times to get various file sizes.
r725 start indexing nicely right after going under 2Gb file size.

Update 2: The issue is no longer present in new ffms2 build (http://forum.doom9.org/showthread.php?p=1607732).

[Myrsloik]

I've released R17 which is hugely improved in most ways. Mostly in that I've tracked down the bugs in avisource which was released a bit before it was ready. It also has a new subtitle filter based on libass contributed by lachs0r.

As usual a slightly longer blog post can be found here (http://www.vapoursynth.com/2012/12/r17-a-soft-package/).

MVTools is next on my list of filters to attack but this will be one long project...

[Chikuzen]

I would appreciate some help in implementing equivalents to mt_edge, mt_inpand, mt_expand, mt_inflate and mt_deflate.
https://github.com/chikuzen/tweak

[Myrsloik]

Interesting, I thought I'd made interleave identical. I think my way is more intuitive that you always get (number of clips)*(longest clip) frames back. I think I won't change it unless someone can think of a very good reason. Small avisynth differences isn't such a big issue in general.

I also fixed the documentation so it will appear correct in r18.

[kolak]

Is it possible to have overlay timecode filter (maybe using just implemented subtitles filter), with simple size and position options.
Is timecode implemented as metadata already?
It would be nice if we could make it visible and have ability to overwrite/specify start TC (and have drop-frame TC also).

[Myrsloik]

Is it possible to have overlay timecode filter (maybe using just implemented subtitles filter), with simple size and position options.
Is timecode implemented as metadata already?
It would be nice if we could make it visible and have ability to overwrite/specify start TC (and have drop-frame TC also).

It should be possible to do once lachs0r actually gets around to compiling a windows version of the subtitle filter. Libass is a bit annoying so I'd rather avoid compiling that stuff.

Currently source filters only add frame duration but I plan to add more frame information to both FFMS2 and AVISource in the future. Such as the original absolute timestamp.

[Gavino]

Interesting, I thought I'd made interleave identical. I think my way is more intuitive that you always get (number of clips)*(longest clip) frames back.
Your way also seems more intuitive to me and is what the Avisynth documentation actually implies.
However, the code does it the other way and this seems to be what the original author intended - see discussion here.

[cretindesalpes]

I think I won't change it unless someone can think of a very good reason.

Typical case:

a = SelectEvery (3, 0)
b = SelectEvery (3, 1)
c = SelectEvery (3, 2)
# Some processing on a, b or c, then:
Interleave (a, b, c)

You generally expect Interleave to return the same number of frames as the input clip.

[Myrsloik]

Typical case:

a = SelectEvery (3, 0)
b = SelectEvery (3, 1)
c = SelectEvery (3, 2)
# Some processing on a, b or c, then:
Interleave (a, b, c)

You generally expect Interleave to return the same number of frames as the input clip.

That's a good reason, I'll make the used behavior an option.

[qyot27]

On 32-bit Linux:
$ ./waf build
Waf: Entering directory `/home/qyot27/vapoursynth/build'
[ 2/21] asm: src/core/asm/expr.asm -> build/src/core/asm/expr.asm.1.o
[ 2/21] asm: src/core/asm/transpose.asm -> build/src/core/asm/transpose.asm.1.o
[ 3/21] asm: src/core/asm/x86check.asm -> build/src/core/asm/x86check.asm.1.o
[ 4/21] asm: src/core/asm/x86cpu.asm -> build/src/core/asm/x86cpu.asm.1.o
[ 5/21] qxx: src/core/cachefilter.cpp -> build/src/core/cachefilter.cpp.1.o
[ 6/21] qxx: src/core/cpufeatures.cpp -> build/src/core/cpufeatures.cpp.1.o
[ 7/21] qxx: src/core/exprfilter.cpp -> build/src/core/exprfilter.cpp.1.o
../src/core/asm/expr.asm:323: error: elf: invalid relocation (WRT or size)
Waf: Leaving directory `/home/qyot27/vapoursynth/build'
Build failed
-> task in 'objs' failed (exit status 1):
{task 154408300: asm expr.asm -> expr.asm.1.o}
['/usr/local/bin/yasm', '-w', '-Worphan-labels', '-Wunrecognized-char', '-Dprogram_name=vs', '-DARCH_X86_64=0', '-felf32', '-I/home/qyot27/vapoursynth/build/include', '-I/home/qyot27/vapoursynth/include', '-I/usr/include/qt4', '-I/usr/include/qt4/QtCore', '../src/core/asm/expr.asm', '-o', 'src/core/asm/expr.asm.1.o']

Reverting r398 makes it compile again.

[Myrsloik]

Sigh... left in some junk from experimenting...
Will unbreak it tomorrow. Writing asm that works everywhere certainly is annoying.

[sl1pkn07]

works on arch64 (r404)

http://paste.ubuntu.com/1481082

patch "fixes" a few warnings http://paste.ubuntu.com/1481085

cython 0.17.2
python 3.3.0
yasm 1.2.0
waf 1.7.8
qt 4.8.4
ffmpeg 1.0.1
libass-git 7e4553

if help anything

[jackoneill]

http://paste.ubuntu.com/1481082

patch "fixes" a few warnings http://paste.ubuntu.com/1481085


Regarding the qFatal() calls: why are you eating the spaces?

[Myrsloik]

Regarding the qFatal() calls: why are you eating the spaces?

And why do you hate empty strings in python? I'll apply the qfatal stuff with spaces tomorrow but the rest is rejected. The precedence of logic operators is also well defined so I don't intend to change it just to please a very high compiler warning level.

[jackoneill]

And why do you hate empty strings in python? I'll apply the qfatal stuff with spaces tomorrow but the rest is rejected. The precedence of logic operators is also well defined so I don't intend to change it just to please a very high compiler warning level.

I think both gcc and clang warn about those empty strings, but they do the job anyway.

[Myrsloik]

I'm finally going completely crazy. I have narrowed down the python memory leak quite a bit but I just can't explain the last part. If anyone happens to be good at debugging python (perhaps you've even used cython too) memory leaks and want to help please contact me. I have a fairly simple case that can reproduce the issue but I simply don't know what triggers it in the first place.

Unless this bug is found it will never be possible to simply press F2 again and again in vdub to refresh a script.

[Adub]

quick update for Waf version.

Index: bootstrap.py
===================================================================
--- bootstrap.py (revision 405)
+++ bootstrap.py (working copy)
@@ -2,7 +2,7 @@

import os, stat, urllib2

-f = urllib2.urlopen('http://waf.googlecode.com/files/waf-1.7.6')
+f = urllib2.urlopen('http://waf.googlecode.com/files/waf-1.7.8')

with open('waf', 'wb') as waf:
waf.write(f.read())


BTW, Myrsloik, do you have a preferred method in which we send patches to you?

[Myrsloik]

quick update for Waf version.

Index: bootstrap.py
===================================================================
--- bootstrap.py (revision 405)
+++ bootstrap.py (working copy)
@@ -2,7 +2,7 @@

import os, stat, urllib2

-f = urllib2.urlopen('http://waf.googlecode.com/files/waf-1.7.6')
+f = urllib2.urlopen('http://waf.googlecode.com/files/waf-1.7.8')

with open('waf', 'wb') as waf:
waf.write(f.read())


BTW, Myrsloik, do you have a preferred method in which we send patches to you?

I'm not that picky but if you open an issue for it on the bug tracker and add it as an attachment I definitely won't miss it. I'm happy as long as it's in some kind of diff/patch format, really.

Anyway, applied.

[Adub]

Cleanup for some of the invert_example.c comments: https://gist.github.com/4450489

[Myrsloik]

Cleanup for some of the invert_example.c comments: https://gist.github.com/4450489

Applied. I've also finally managed to solve most of the evils of python so the next version will leak a lot less memory.

[Adub]

Applied. I've also finally managed to solve most of the evils of python so the next version will leak a lot less memory.

Excellent, good to hear. I've begun work on my CUDA environment. I'm getting used to the Waf build system and incorporating the CUDA compiler into the mix. It's almost up an ready for dev, but I've got a few CFLAGS issues to fix first.

[Myrsloik]

Will the missing operator ^(power) in Expr be available in the next version as well? :D

I have no optimized code for it so just use exp and log to get the same result.

[Reel.Deel]

I have no optimized code for it so use exp and log to get the same result.

For a RPN/LUT dummy like myself, how would I write this mt_lut expression using Expr?
mt_lut("x 128 - 1.49 * x 128 - 2 ^ x 128 - 2 ^ 9 + / * 128 +")

I also think it would be a good idea for existing mt_lut/x/y/z expressions to be compatible with Expr with no additional gimmicks.

[Myrsloik]

For a RPN/LUT dummy like myself, how would I write this mt_lut expression using Expr?
mt_lut("x 128 - 1.49 * x 128 - 2 ^ x 128 - 2 ^ 9 + / * 128 +")

I also think it would be a good idea for existing mt_lut/x/y/z expressions to be compatible with Expr with no additional gimmicks.

Well... theoretically I could add the ^ operator but it would do you no good since it's very slow just for squaring things. In this case x*x is much faster. So I give you this equivalent that doesn't even need ^ and removes some duplicate evaluation:
"x 128 - 1.49 * x 128 - dup * dup 9 + / * 128 +"
(warning, completely untested but should work and if it doesn't just tell me and I'll actually try it myself)

When you go from a lut to actually evaluating the expression for every pixel you have to be more careful about how you write it too or it could become slow.

Bonus material by jfs, the original expression in readable form:
(x-128)*1.49 * (((x-128)^2) / ((x-128)^2 + 9)) + 128

[Reel.Deel]

(warning, completely untested but should work and if it doesn't just tell me and I'll actually try it myself)
Expr expression works as expected, the only only thing is that mt_lut by default only processes the luma. So I read the Expr documentation and added the following and now the results look identical.
avs = core.avs.mt_lut(c1=src, expr="x 128 - 1.49 * x 128 - 2 ^ x 128 - 2 ^ 9 + / * 128 +", Y=3, U=1, V=1)
vs = core.std.Expr(clips=[src], expr=["x 128 - 1.49 * x 128 - dup * dup 9 + / * 128 +", "", ""])

Thanks for the bonus material. Much easier to understand. :)

[wOxxOm]

Well... theoretically I could add the ^ operator but it would do you no good since it's very slow just for squaring things. In this case x*x is much faster. So I give you this equivalent that doesn't even need ^ and removes some duplicate evaluation
What about "something 1.05 ^"?
What about "something something ^"?
Also the calculation is done only once while building lut table, so the overhead may safely be neglected.

I also think it would be a good idea for existing mt_lut/x/y/z expressions to be compatible with Expr with no additional gimmicks.
Absolutely.
There are lots of useful scripts with very complex expressions.

[pie]

What about "something 1.05 ^"?
What about "something something ^"?

I'm not too sure with RPN, but something like this?
something log 1.05 * exp

For a^b
a log b * exp

--> e^(ln(a)*b)

[Myrsloik]

what about "something 1.05 ^"?


Absolutely.
There are lots of useful scripts using very complex expressions.

Then you have to remember from school that a^x = e^(x*ln(a))

so you rewrite it to:
a log x * exp (or something log 1.05 * exp)

And I have to reiterate that just copying and pasting expressions is kind of a bad idea for performance reasons. Just take the one I changed around above as an example.

[wOxxOm]

just copying and pasting expressions is kind of a bad idea for performance reasons.
What performance?

the calculation is done only once while building lut table, so the overhead may safely be neglected.
This is not something one learns at school, I guess, but still it's obvious.

[Chikuzen]

lutxyz requires at least about 16MiB(256*256*256*1byte).
Since a huge table is not settled in CPU cache, it may need very very slow memory access each time.
Therefore, runtime calcuration is often faster than LUT.

[wOxxOm]

Chikuzen, is this somehow related to my remarks about the majority of lut scripts, taking in account that lutxyz should be avoided at all costs and thankfully not commonly used anyway? Also, don't forget that memory access penalty during building of that 16MB table is *nothing* compared with memory penalty during processing of lots of actual video frames.

[StainlessS]

It is lut access during processing of video frames that will have a heavy cache miss penalty.

[wOxxOm]

StainlessS, I wrote about the same earlier, and to reiterate my point I'll rephrase: having a cpu-heavy instructions in expressions (like "^" - power) takes only an insignificant amount of overall processing time (it's calculated only one time, not each frame) so there's no need to exclude those in favour of non-practical 'performance gains' while degrading substantially ease of porting for existing lut-scripts.

[TheFluff]

Also the calculation is done only once while building lut table, so the overhead may safely be neglected.
expr() in Vapoursynth doesn't use a LUT, it re-evaluates the expression for each pixel, hence Myrsloik's comments about how you should be careful with your RPN expressions. It's done that way because building LUT's for high bitdepth video is a really stupid idea.

People who like faith-based optimizations may note that expr() isn't really any significantly slower than mt_lutxy[z] in many cases; for simple expressions it can even be faster than mt_lutxy.

[wOxxOm]

expr() in Vapoursynth doesn't use a LUT, it re-evaluates the expression for each pixelEven for 8-bit videos? Oh, quite unexpected... and seems errr... inefficient in this particular case.

And what about expr with complex expression for 1 video comparing with mt_lut (not xy) ?

[Myrsloik]

Even for 8-bit videos? Oh, quite unexpected... and seems errr... inefficient.

1. It's not inefficient, ironically x y + is about the same speed at mt_lutxy since memory reads for a lookup isn't magic
2. You can have 3 inputs (or more) without being inefficient
3. You can have mixed bitdepth inputs
4. There's a reason why the Lut and Lut2 filters exist
5. Get a clue, faith based criticism just annoys me and all other competent people reading this forum

[TheFluff]

And what about expr with complex expression for 1 video comparing with mt_lut (not xy) ?
If you want a LUT, you can have a LUT, see the standard library functions Lut() and Lut2(). Of course, they may or may not actually be faster depending on what you're doing and they have some restrictions that expr() doesn't have.

[wOxxOm]

Peace. I thought we were discussing technical issues not meant to be taken personally. Thank you for your answers and patience anyway.

[mastrboy]

# ret will contain frames 6 to 9 (unlike trim the end value of python slicing is not inclusive)
ret = clip[6:10]

Now that is just illogical, why is it not inclusive?

[Myrsloik]

# ret will contain frames 6 to 9 (unlike trim the end value of python slicing is not inclusive)
ret = clip[6:10]

Now that is just illogical, why is it not inclusive?

1. No, it's actually very common. Just see vdub and its timeline.
2. It's how the slicing operator works for every other sanely implemented python type so doing it differently would be even more "illogical".
3. This: a[:3]+b[3:10]+c[10:] (the point being that you can mindlessly set the end frame of the previous to the start of the first.

[kolak]

Very well known "problem" with different NLEs, transcoders, etc
Not including last frame is the most common approach as far as I can tell.

[gpower2]

In case you haven't noticed, latest VirtualDub (1.10.3) added support for .vpy extension!

[Adub]

Quick update on my end:

Fixed CFLAGS issues when comiling with Waf for CUDA support. I've added a bit more functionality to the Waf script to make everything work cleanly, although it might need some touch ups here and there. Currently, I've begun work porting some example filters, so that I can determine the best way to approach some of the algorithmic ports to the massively multithreaded CUDA architecture.

I'll be working with my professor to get remote access to some more "special" CUDA machines that we have available, which should vastly improve my productivity as I can then work remotely.

[Adub]

https://gist.github.com/4513971

Add dependency check for libavcodec, as vsresize.c requires the avcodec.h header.

[Myrsloik]

https://gist.github.com/4513971

Add dependency check for libavcodec, as vsresize.c requires the avcodec.h header.

Applied.

I'm a bit curious about your CUDA project. Did you get anything working yet?

[sl1pkn07]

fall build

NameError: global name 'AVCODEC' is not defined

http://paste.ubuntu.com/1522187

greetings

edit: missing ' ' in line 298

[Are_]

--- ../src/vapoursynth-read-only/wscript 2013-01-12 04:15:58.963414147 +0100
+++ wscript 2013-01-12 04:16:42.673431040 +0100
@@ -295,7 +295,7 @@

bld(features = 'c qxx asm',
includes = 'include',
- use = ['QTCORE', 'SWSCALE', 'AVUTIL', AVCODEC],
+ use = ['QTCORE', 'SWSCALE', 'AVUTIL', 'AVCODEC'],
source = bld.path.ant_glob(sources),
target = 'objs')

@@ -307,7 +307,7 @@

if bld.env.STATIC == 'true':
bld(features = 'c qxx asm cxxstlib',
- use = ['objs', 'QTCORE', 'SWSCALE', 'AVUTIL'],
+ use = ['objs', 'QTCORE', 'SWSCALE', 'AVUTIL', 'AVCODEC'],
target = 'vapoursynth',
install_path = '${LIBDIR}')

[sl1pkn07]

zankius for the patch.

[Adub]

Hmm, that's odd. My patch had proper quoting on the AVCODEC. Don't know why it got lost in translation.

I did miss the static line though.


As for my CUDA work, I'm still pre-alpha at this point. I have finished integrating the cuda 'nvcc' compiler into the Waf build system, which took a little while to get right.

I'm working on doing an initial test port of the invert filter, just to begin testing. I'm currently in the design phase, as I want to get the kernel "intelligent" enough before I start expanding the scope of the cuda port process.

The biggest difference is how the CUDA threading architecture is designed, so I'm figuring out the best way to port over the processing of a multitude of planes.

At the same time I'm learning the internal architecute of the Vapoursynth API, and Core. I don't want to have to implement anything that's been done already that I just didn't know about.

But I have finally received remote access to a Tesla-based machine, so I can work from anywhere now.

Unfortunately, I can only work on this project a few hours a day, as school has resumed.

[Myrsloik]

New version. Big bugfix release this time. As usual the changelog is in the first post and you can find a blog post about it here: R18 – Beta Quality (http://www.vapoursynth.com/2013/01/r18-beta-quality/)

[06_taro]

Great job for those fixes on memory leaks!

btw, version() still says r17 in r18: VapourSynth Video Processing Library
Copyright (c) 2012 Fredrik Mellbin
Core r17
API r3

[sl1pkn07]

and update files in TAG svn :P

[Myrsloik]

and update files in TAG svn :P

Yawn. I'm just going to let this single issue be. I think I got everything else right.

[gpower2]

Finally managed to run a comparison test myself.
I used 1500 frames of a 720p mp4 video downloaded from youtube (http://www.youtube.com/watch?v=y-9DVU6Kux8)

Software used:
VapourSynth R18
VirtualDub 1.10.3
XviD 1.3.2 vfw (koepi build)
AviSynth 2.5.8 (official vanilla version from http://www.avisynth.org)
Avisynth Core Filters (from http://www.vapoursynth.com)
GenericFilters 0.4.0 (from http://forum.doom9.org/showthread.php?t=166842)
HAvsFunc r3 (from http://forum.doom9.org/showthread.php?t=166582)
FFMS2 r739 (from http://forum.doom9.org/showthread.php?p=1607732#post1607732)

Scripts tested:
AviSynth
FFVideoSource(source = "C:\Garslan\Videos\Videoclips\1990-2000\Babybird - Atomic Soda ( HD ).mp4",
track = -1, cache = true, cachefile = "C:\Garslan\Videos\Videoclips\1990-2000\Babybird - Atomic Soda ( HD ).mp4.ffindex",
fpsnum = -1, fpsden = 1, pp="", threads = -1,
timecodes = "C:\Garslan\Videos\Videoclips\1990-2000\Babybird - Atomic Soda ( HD ).mp4.tcodes.txt", seekmode = 1)
trim(0,1500)
lsfmod()

VapourSynth - SingleThread
import vapoursynth as vs
import sys
import havsfunc as hf
core = vs.Core(threads=1)
haf = hf.HAvsFunc(core)

core.std.LoadPlugin(path=r'D:\Downloads\ffms2.dll')
core.std.LoadPlugin(path=r'C:\vsplugins\genericfilters.dll')
core.std.LoadPlugin(path=r'C:\Program Files (x86)\VapourSynth\filters\temporalsoften.dll')
core.std.LoadPlugin(path=r'C:\Program Files (x86)\VapourSynth\filters\eedi3.dll')

core.avs.LoadPlugin(path=r'C:\vsplugins\avisynthfilters.dll')
core.avs.LoadPlugin(path=r'C:\Program Files (x86)\AviSynth 2.5\plugins\mt_masktools-26.dll')
core.avs.LoadPlugin(path=r'C:\Program Files (x86)\AviSynth 2.5\plugins\RemoveGrain_v10pre1.dll')

ret = core.ffms2.Source(source=r'C:\Garslan\Videos\Videoclips\1990-2000\Babybird - Atomic Soda ( HD ).mp4')
ret = core.std.Trim(ret, 0, 1500)
ret = haf.LSFmod(ret)

last = ret

VapourSynth - Multithread
import vapoursynth as vs
import sys
import havsfunc as hf
core = vs.Core(threads=0)
haf = hf.HAvsFunc(core)

core.std.LoadPlugin(path=r'D:\Downloads\ffms2.dll')
core.std.LoadPlugin(path=r'C:\vsplugins\genericfilters.dll')
core.std.LoadPlugin(path=r'C:\Program Files (x86)\VapourSynth\filters\temporalsoften.dll')
core.std.LoadPlugin(path=r'C:\Program Files (x86)\VapourSynth\filters\eedi3.dll')

core.avs.LoadPlugin(path=r'C:\vsplugins\avisynthfilters.dll')
core.avs.LoadPlugin(path=r'C:\Program Files (x86)\AviSynth 2.5\plugins\mt_masktools-26.dll')
core.avs.LoadPlugin(path=r'C:\Program Files (x86)\AviSynth 2.5\plugins\RemoveGrain_v10pre1.dll')

ret = core.ffms2.Source(source=r'C:\Garslan\Videos\Videoclips\1990-2000\Babybird - Atomic Soda ( HD ).mp4')
ret = core.std.Trim(ret, 0, 1500)
ret = haf.LSFmod(ret)

last = ret

I encoded the clip with XviD default parameters and the performance results are:

Run 1:

AviSynth Ellapsed time: 79587ms / 00:01:19.5870000
VapourSynth SingleThread Ellapsed time: 153168ms / 00:02:33.1680000
VapourSynth Multithread Ellapsed time: 81528ms / 00:01:21.5280000

Run 2:
AviSynth Ellapsed time: 80519ms / 00:01:20.5190000
VapourSynth SingleThread Ellapsed time: 153752ms / 00:02:33.7520000
VapourSynth Multithread Ellapsed time: 84267ms / 00:01:24.2670000

AviSynth beat VapourSynth in single thread mode hands down. It even beat VapourSynth in multithread mode.
I guess the heavy use of imports and mixed filters doesn't make VapourSynth usable for "production" level scripts.

Next time, I'll try a simple script, with no filters, and a real script (if I can manage to port it to VapourSynth)
and see the results.

P.S.
In VirtualDub, pressing F2 does reloads the script, but it doesn;t reload the whole python environment, meaning the imports etc.
Try to load a script with imports, and then comment them out from the script and press F2. It should not work, yet it does...

Still, I believe that VapourSynth can become the new standard, if the most popular AviSynth filters are ported to it.
And I mean natively, not with python scripts...

Keep up the good work! :D

[Myrsloik]

Uh... did you run the comparison using only avisynth filters? Because that's what's really important. That and if you use eedi3 force the filter itself to be single threaded. And how many cores does your cpu have?

As for cleaning out the python environment it already is being done almost as much as is possible. It's simply designed as having global state to be annoying, or so my theory goes.

[gpower2]

Well, I certainly used mixed filters, I mean, I only wanted to test LSFMod, which I use a lot with AviSynth, but to use it with VapourSynth, you need a lot of dependencies.
And since it's an avsi avisynth script, I can't import it directly in VapourSynth, correct? (I thought I saw someone wrote an avisynth script parser for VapourSynth, but I didn't find it now)

The test pc in my case has a Q9505 Core2 Quad-Core CPU @ 2.83GHz and 8GB RAM.

Do you have some particular test case in mind?

I also have a 930 i7 and 18GB RAM at home, so I could also test something there...

[Myrsloik]

The test case has to be like this:
1. Only use avisynth filters, both in avisynth and vs so it's exactly the same filter code running. It's the core performance I want to compare.
2. Make sure all filters in it don't secretly do multithreaded stuff, eedi3 especially so
3. Any simple avisynth script will do as a start. It may even be better to start small and only do ffms.Source() and avisource.avisource() and then adding things on to that.

Those are my guidelines for testing. (yes, I know that vs avisource obviously is slightly different but it should perform identically)

[kolak]

All these speed issues with avs filters are not related to prefetching problem?

http://forum.doom9.org/showthread.php?p=1610787#post1610787

[Myrsloik]

All these speed issues with avs filters are not related to prefetching problem?

http://forum.doom9.org/showthread.php?p=1610787#post1610787

Prefetching is the biggest problem at least. I think.

[[ReX]]

I spotted typo on Adub's patch to invert_example.c.

// Once all frames are ready, the the filter will be called with arAllFramesReady. It is now time to

Not that his patch added it, there's where I saw it. ;)

[Adub]

Damn [Rex], you've got a good eye. I don't know how I missed that 'the' while I was combing the source code. I was purposefully looking for that very thing!

[gpower2]

Made another performance test, as requested by Myrsloik.

PC Used
Q9505 Core2 Quad-Core CPU @ 2.83GHz, 8GB RAM, Win7 x64 Enterprise

Software Used
VapourSynth R18
VirtualDub 1.10.3
XviD 1.3.2 vfw (koepi build)
AviSynth 2.5.8 (official vanilla version from http://www.avisynth.org)
FFMS2 r739 (from http://forum.doom9.org/showthread.ph...32#post1607732)

Scripts tested

With 720p mp4 video:

AviSynth - SingleThread FFMS2
LoadCPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\ffms2.dll")
FFVideoSource(source = "Babybird - Atomic Soda ( HD ).mp4", track = -1, cache = true, \
cachefile = "Babybird - Atomic Soda ( HD ).mp4.ffindex", \
fpsnum = -1, fpsden = 1, pp="", threads = 1, timecodes = "Babybird - Atomic Soda ( HD ).mp4.tcodes.txt", seekmode = 1)

AviSynth - MultiThread FFMS2
LoadCPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\ffms2.dll")
FFVideoSource(source = "Babybird - Atomic Soda ( HD ).mp4", track = -1, cache = true, \
cachefile = "Babybird - Atomic Soda ( HD ).mp4.ffindex", \
fpsnum = -1, fpsden = 1, pp="", threads = -1, timecodes = "Babybird - Atomic Soda ( HD ).mp4.tcodes.txt", seekmode = 1)

VapourSynth SingleThread -SingleThread FFMS2
import vapoursynth as vs
import sys
core = vs.Core(threads = 1)
core.std.LoadPlugin(path = r'C:\Program Files (x86)\AviSynth 2.5\plugins\ffms2.dll')
ret = core.ffms2.Source(source = r'Babybird - Atomic Soda ( HD ).mp4',
track = -1, cache = 1,
cachefile = r"Babybird - Atomic Soda ( HD ).mp4.ffindex",
fpsnum = -1, fpsden = 1, threads = 1, seekmode = 1,
timecodes = r"Babybird - Atomic Soda ( HD ).mp4.tcodes.txt")
last = ret

VapourSynth SingleThread - MultiThread FFMS2
import vapoursynth as vs
import sys
core = vs.Core(threads = 1)
core.std.LoadPlugin(path = r'C:\Program Files (x86)\AviSynth 2.5\plugins\ffms2.dll')
ret = core.ffms2.Source(source = r'Babybird - Atomic Soda ( HD ).mp4',
track = -1, cache = 1,
cachefile = r"Babybird - Atomic Soda ( HD ).mp4.ffindex",
fpsnum = -1, fpsden = 1, threads = -1, seekmode = 1,
timecodes = r"Babybird - Atomic Soda ( HD ).mp4.tcodes.txt")
last = ret

VapourSynth MultiThread - SingleThread FFMS2
import vapoursynth as vs
import sys
core = vs.Core(threads = 0)
core.std.LoadPlugin(path = r'C:\Program Files (x86)\AviSynth 2.5\plugins\ffms2.dll')
ret = core.ffms2.Source(source = r'Babybird - Atomic Soda ( HD ).mp4',
track = -1, cache = 1,
cachefile = r"Babybird - Atomic Soda ( HD ).mp4.ffindex",
fpsnum = -1, fpsden = 1, threads = 1, seekmode = 1,
timecodes = r"Babybird - Atomic Soda ( HD ).mp4.tcodes.txt")
last = ret

VapourSynth MultiThread - MultiThread FFMS2
import vapoursynth as vs
import sys
core = vs.Core(threads = 0)
core.std.LoadPlugin(path = r'C:\Program Files (x86)\AviSynth 2.5\plugins\ffms2.dll')
ret = core.ffms2.Source(source = r'Babybird - Atomic Soda ( HD ).mp4',
track = -1, cache = 1,
cachefile = r"Babybird - Atomic Soda ( HD ).mp4.ffindex",
fpsnum = -1, fpsden = 1, threads = -1, seekmode = 1,
timecodes = r"Babybird - Atomic Soda ( HD ).mp4.tcodes.txt")
last = ret

With 288p mp4 video:
The same as above, with different source file.

With 320p avi video:
AviSynth - AviSource
AviSource("C:\Garslan\Videos\Videoclips\2001-2009\Banned, Uncensored & Uncut Music Videos\Madonna - Erotica(Banned Version).avi")

VapourSynth SingleThread - VS avisource
import vapoursynth as vs
import sys
core = vs.Core(threads = 1)
core.std.LoadPlugin(path = r'C:\Program Files (x86)\VapourSynth\filters\avisource.dll')
ret = core.avisource.AVISource(r'C:\Garslan\Videos\Videoclips\2001-2009\Banned, Uncensored & Uncut Music Videos\Madonna - Erotica(Banned Version).avi')
last = ret

VapourSynth MultiThread - VS avisource
import vapoursynth as vs
import sys
core = vs.Core(threads = 0)
core.std.LoadPlugin(path = r'C:\Program Files (x86)\VapourSynth\filters\avisource.dll')
ret = core.avisource.AVISource(r'C:\Garslan\Videos\Videoclips\2001-2009\Banned, Uncensored & Uncut Music Videos\Madonna - Erotica(Banned Version).avi')
last = ret


Results

AviSynth - SingleThread FFMS2 [720p]: Ellapsed time: 269908ms / 00:04:29.9080000
AviSynth - MultiThread FFMS2 [720p]: Ellapsed time: 277995ms / 00:04:37.9950000
VapourSynth MultiThread - MultiThread FFMS2 [720p]: Ellapsed time: 282046ms / 00:04:42.0460000
VapourSynth MultiThread - SingleThread FFMS2 [720p]: Ellapsed time: 275250ms / 00:04:35.2500000
VapourSynth SingleThread -SingleThread FFMS2 [720p]: Ellapsed time: 271664ms / 00:04:31.6640000
VapourSynth SingleThread - MultiThread FFMS2 [720p]: Ellapsed time: 280996ms / 00:04:40.9960000

AviSynth - SingleThread FFMS2 [288p]: Ellapsed time: 16168ms / 00:00:16.1680000
AviSynth - MultiThread FFMS2 [288p]: Ellapsed time: 19572ms / 00:00:19.5720000
VapourSynth MultiThread - SingleThread FFMS2 [288p]: Ellapsed time: 16548ms / 00:00:16.5480000
VapourSynth MultiThread - MultiThread FFMS2 [288p]: Ellapsed time: 17411ms / 00:00:17.4110000
VapourSynth SingleThread -SingleThread FFMS2 [288p]: Ellapsed time: 16192ms / 00:00:16.1920000
VapourSynth SingleThread - MultiThread FFMS2 [288p]: Ellapsed time: 17246ms / 00:00:17.2460000

AviSynth - AviSource [320p]: Ellapsed time: 27360ms / 00:00:27.3600000
VapourSynth MultiThread - VS avisource [320p]: Ellapsed time: 28255ms / 00:00:28.2550000
VapourSynth SingleThread - VS avisource [320p]: Ellapsed time: 27361ms / 00:00:27.3610000

Singlethread VapourSynth doesn't seem to reach singlethread AviSynth performance, while multithread VapourSynth barely makes it.

I used the same ffms2.dll for both AviSynth and VapourSynth.
Encoded via VFW interface with VirtualDub and XviD.

[sven_x]

I think for performance testing one should chose the fastest encoder that is available (or even no encoding). Otherwise most of the CPU usage will be used for encoding.

[gpower2]

I think for performance testing one should chose the fastest encoder that is available (or even no encoding). Otherwise most of the CPU usage will be used for encoding.

Well, there is also the HDD bottleneck you should take in mind. ;)

Also, the results are quite consistent, so I don't think that using any other encoder would make any difference, apart from the different numbers...

[kolak]

I don's see any big difference. You could also note CPU and RAM usage.
Also- all what you have tested is mainly ffms2 decoding performance or encoding speed. You could simply run analysis pass in Vdub to get just processing speed (no encoding).

[Adub]

How is everyone testing the higher bit-depth functionality of Vapoursynth? I'd love to get some samples and/or a workflow for testing input above 8-bits.

[Myrsloik]

How is everyone testing the higher bit-depth functionality of Vapoursynth? I'd love to get some samples and/or a workflow for testing input above 8-bits.

Either using the resizer to convert random 8bit junk to 10 or 16bit or with blankclip. I suppose ffms2 and 10bit h264 files would do as well...

[kolak]

I tested quite few- ProRes, DNxHD, v210 , AVC-I with dithering, resizing and all works fine.

[Adub]

Does ffms2 support up to 16-bit input?

If so, Kolak, what bit-depths are the clips that you are testing with? Do they go up to 16-bit? Would you mind posting a few samples for me to download?

[kolak]

Yes, as far as I know ffms2 will read 16bit files. My samples are mostly 10bit. Can't share, most with copyrights.
You can find some ProRes etc files on the web, eg.

http://www.blackmagicdesign.com/support/detail?sid=3948&pid=11735&leg=false&cp=true&os=win

Alexa ProRes files in LOG color space, so you can also play with LUT files to get proper colors :)

VS also has support for b48r format (16bit RGB), which is used inside MOV- it's introduced as a part of some bigger idea, which may become reality one day :)

[qyot27]

From my tests with high bit depth H.264 (12- and 14-bit 4:2:0, 4:2:2, and 4:4:4), FFMS2 opens them just fine, VapourSynth passes them through as 16-bit to y4m, and as long as the build of mpv/x264/ffmpeg/whatever can accept the extended y4m format, you're pretty much golden on that front. May or may not need stream fixing with ffmpeg as a middle step, though (remember, the extended y4m is still experimental; you've gotta use -strict -1 for it to pipe them through).

You just need to make sure the build of FFMS2 is using mainline FFmpeg as the library backend, since the fork doesn't support >10-bit for H.264 (and for all I know, it doesn't support >10 & <16 in rawvideo either; there was also a bug report for mpv that showed as much).

The aforementioned samples. Six files: 4:2:0, 4:2:2, and 4:4:4 in 12-bit and 14-bit. The source actually came from an editing project, so no audio (it wouldn't matter, either).
http://www.mediafire.com/?9jcjn111habg32x

You can verify the bit depth/colorspace using MediaInfo, and play them directly using any fairly recent build of mpv (at least). I do also have RGB samples in 12- and 14-bit H.264, but either they weren't generated correctly, or all the kinks haven't been hammered out of FFmpeg's decoding paths for those cases yet (which is what I think is the more likely scenario). They'll play, and the colorspace is recognized as gbrp[12/14]le, but they're a deep pink/green instead of the correct colors. That's why I didn't include them in the 7z.

[sneaker_ger]

You could also get 16 bit PNGs of Sintel for example:
http://media.xiph.org/sintel/

Works fine with Chikuzen's image reader plugin.

[Adub]

Woot! Success!

I just succesfully tested my CUDA enhancments, and they work!! Had some trouble with some damn integer truncation for a bit, as well as pitch calculation, but I re-learned middle school really quick. :P

It's still currently slower than CPU, but I completely expected that. I'm going to start integrating more of the GPU work into the core, which will offer a more seamless dev experience and less work in each filter.

So as of now, the invert filter works. I'm going to streamline a lot of code, and look at porting more filters, but I just wanted to report initial success (finally).

[LoRd_MuldeR]

I got a question:

The recommended way to encode from VapourSynth with x264 seems to be using a pipe:
"python script.py | x264 --demuxer y4m --output encodedfile.mkv -"

But: Unless we we properly set "--frames N", x264 will not be able to display progress when reading from a pipe.

So what would be the most simple way to programmatically detect the correct frame count from a given VapourSynth script file?

For Avisynth I would open the script with avs2yuv and then parse the text output.

I guess the same thing could be done with Python. But I'm not very good at Python. A small example would be appreciated! :)


What I would need is something like:
Python.exe "detect_framecount.py" "C:\Some Path\Script we are going to pipe into x264.py"

And ideally it would write something like this to the STDOUT:
Number of frames: 12345

[sl1pkn07]

for example, in my encode scripts with avisynth in my linux, call avs2pipe before encode and extract number of frames and WxH from output and parse to x264. and with avs2yuv (asv2pipe and avs2pipemod don't silence his output) process the video


x264="/path/of/x264/
cmd="$(<x264_commands)"
info="$(wine avs2pipe --info "$1")"
frames="$(echo -n "$info" | sed -ne 's/^v:frames\(.*\)$/\1/p' | sed 's|[\r ]||g')"
res+="$(echo -n "$info" | sed -ne 's/^v:width\(.*\)$/\1/p' | sed 's|[\r ]||g')"
res+="x"
res+="$(echo -n "$info" | sed -ne 's/^v:height\(.*\)$/\1/p' | sed 's|[\r ]||g')"

wine avs2yuv "$1" -o - | "$x264" - --output "$2" --stdin y4m --frames "$frames" --input-res "$res" $cmd


./encode archive.avs vid.mkv

(x264_commands: is a file with x264 parmeters for encode)

i supposed with vapoursynth its the same method, but swap "wine avs2yuv" to "python script.py" command

in win make a .bat with same process and running xd

greetings

[ChaosKing]

detect_framecount.py
This will print the number of frames :

import vapoursynth as vs
import sys
core = vs.Core(threads = 6)
c = core.avisource.AVISource(r'e:\P4_14.avi')
print("Number of frames: ", c.num_frames) # or just print(c.num_frames)




And you can also do thinks like this:

import vapoursynth as vs
import sys
import os
core = vs.Core(threads = 6)
c = core.avisource.AVISource(r'e:\P4_14.avi')
print("Number of frames: ", c.num_frames) # or just print(c.num_frames)
os.system("calc") # or your x264 call...


But I'm not a python coder so maybe there is a better way

@sl1pkn07, I think the combination with wine will slow down your encode unnecessarily

[LoRd_MuldeR]

ChaosKing, the problem is that modifying the original script will not be possible/desirable.

What I need is a Python script that will load an existing VapourSynth script (which is provided by the user and which will be piped into x264 as-is) and from that script detect the total frame count.

I know I can use imp.load_source() to load the VPY script, but I don't really know what to do with it...

[Myrsloik]

ChaosKing, the problem is that modifying the original script will not be possible.

What I need is a Python script that will load an existing VapourSynth script (which is provided by the user) and from that script detect the total frame count.

It will kinda improve in the next version/not so distant future. The clip.output() stuff will go since it's just unnecessarily complicated and will be replaced with a standard program to pipe the output. So if you're not in a hurry it should solve itself in a bit. The main problem is that if someone uses clip.output() you have to rewrite the script to get the number of frames...

[LoRd_MuldeR]

It will kinda improve in the next version/not so distant future. The clip.output() stuff will go since it's just unnecessarily complicated and will be replaced with a standard program to pipe the output.

That would be nice :)

Ideally that "standard program" would write the data to the STDOUT and, at the same time, print some basic info (including total frame count) onto the STDERR.

And it would also have a switch to only print the info but not output anything.

So if you're not in a hurry it should solve itself in a bit.

Well it's the thing that's currently blocking me for properly integrating VapourSynth into my x264 GUI. But it surly can wait a few more days.

The main problem is that if someone uses clip.output() you have to rewrite the script to get the number of frames...

Yup, that's the problem. And re-writing is not easy to do, because I cannot expect the user's script to be in a specific form...

[Adub]

Myrsloik, do you have a coding style that you prefer we stick to when hacking around in the core? Similar to libav's coding rules (https://libav.org/developer.html#toc-Code-formatting-conventions)?

[Myrsloik]

Myrsloik, do you have a coding style that you prefer we stick to when hacking around in the core? Similar to libav's coding rules (https://libav.org/developer.html#toc-Code-formatting-conventions)?

Qt inspired with hints of pascal verbosity?

You can probably see the pattern in all the existing code and try to copy it.

[Adub]

Okay, I figured it would be something like that.

I asked because I was considering throwing together a basic indent command line to make sure I didn't miss violations from time to time. I've been following your style as far as I've paid attention to so far, so I just wanted to make sure that any possible merges would work just fine.

[Adub]

Myrsloik, how exactly does the "struct VSAPI" definition in vapoursynth.pxd relate to the "const vsapi" initialization in vsapi.cpp?

I though at first that it was a direct 1:1 mapping between API functions, but now I'm not so sure. Does order not matter when adding a new function to both of these?

For reference, I'm adding a new function, transferVideoFrame(), that can be used by video filters to transfer frame data between the host memory and GPU memory.

Forgive me if this is a stupid question, I just have little familiarity with the Cython integration layer.

Edit I see that the full API is declared in Vapoursynth.h, but I'm still wondering about the .pxd typedef.

[Myrsloik]

Cython is a bit special. The pxd is just says that a function/data structure with something kinda like this or that exists. For the actual compilation cython still includes vapoursynth.h and uses its definitions directly. If you look some more you'll also see that cython doesn't have the const keyword.

In theory its definitions should match vapoursynth.h but it doesn't really matter that much as long as it compiles.

[Adub]

Okay, interesting (and freakin' odd). Thanks for the explanation, Myrsloik! I'll crack on with Vapoursynth.h and vsapi.cpp.

[LoRd_MuldeR]

It will kinda improve in the next version/not so distant future. The clip.output() stuff will go since it's just unnecessarily complicated and will be replaced with a standard program to pipe the output.

Sorry for my impatience, but any news on that yet?

[Myrsloik]

Sorry for my impatience, but any news on that yet?

Not really, real life has intervened. This time I don't even dare to guess when stuff will be done...

[mp3dom]

I'm getting errors while opening a MOV with "Animation RLE" as a codec.
Using FFMS2 (both version, found here: http://forum.doom9.org/showpost.php?p=1607732&postcount=1739) the error is:

Traceback (most recent call last):
File "<pyshell#6>", line 1, in <module>
vid=core.ffms2.Source(source='vid_23976fps.mov')
File "vapoursynthpp.pyx", line 1048, in vapoursynth.Function.__call__ (src\cython\vapoursynthpp.c:15779)
vapoursynth.Error: 'Source: No suitable output format found'


The output format should be (I guess) RGB32.
Using the FFMS2 version compiled also for AviSynth (the second link) in AviSynth, doesn't throw any error and correctly open the file as RGB32.

[Adub]

Hmmm, looking at VapourSynth.h, there doesn't appear to be a corresponding VSPresetFormat for RGB32.

Do you have a small sample of your problem clip?

Edit: Ahh, I see that CompatBGR32 is what is used instead.

[Myrsloik]

I'm getting errors while opening a MOV with "Animation RLE" as a codec.
Using FFMS2 (both version, found here: http://forum.doom9.org/showpost.php?p=1607732&postcount=1739) the error is:

Traceback (most recent call last):
File "<pyshell#6>", line 1, in <module>
vid=core.ffms2.Source(source='vid_23976fps.mov')
File "vapoursynthpp.pyx", line 1048, in vapoursynth.Function.__call__ (src\cython\vapoursynthpp.c:15779)
vapoursynth.Error: 'Source: No suitable output format found'


The output format should be (I guess) RGB32.
Using the FFMS2 version compiled also for AviSynth (the second link) in AviSynth, doesn't throw any error and correctly open the file as RGB32.

This is pure speculation but swscale couldn't do all planar<>packed rgb conversions properly until very recently. Internally all rgb needs to be planar for vs Maybe if you try with a newer compile of ffms2. I could be wrong too...

[Reel.Deel]

Hi Myrsloik, here comes a long post. :)
I recently tested a handful of plugins to see which ones display the following warning.
Avisynth Compat: requested frame xxx not prefetched, using slow method that may deadlock

This is the script I used for all of the plugins tested. All plugins tested with default values, using VapourSynth R18 on WinXP (SP3).
import vapoursynth as vs
core = vs.Core(threads=4)

# Plugins
core.avs.LoadPlugin(path=r'C:\DGDecodeNV.dll')
core.avs.LoadPlugin(path=r'C:\WhateveAVSplugin.dll')

# Source
src = core.avs.DGSource(dgi=r'C:\blu-ray 1080p.dgi')

# For plugins that only work in YUY2
#src = core.resize.Point(clip=src, format=vs.COMPATYUY2)

# Processing
src = core.avs.WhateverName(c1=src)

# VDub ouput
last = src
I then, loaded the VS script using AVISourve in AVS and used AVSMeter (http://forum.doom9.org/showthread.php?t=165528&highlight=AVSMETER) to see if the warning would be displayed.

Here's a list of plugins that need to be added to the prefetch list. (http://code.google.com/p/vapoursynth/source/browse/trunk/src/avisynth/avisynth_compat.cpp#188)

checkmate (http://forum.doom9.org/showthread.php?p=1571520#post1571520)
DeCrawl (http://avisynth.org/warpenterprises/#DeCrawl)
DeCross (http://nullinfo.s21.xrea.com/#DeCross)
DeDot (http://nullinfo.s21.xrea.com/#DeDot_YV12)
DeGrainMedian (http://avisynth.org.ru/degrain/degrainmedian.html)
dw3DNR (http://www5e.biglobe.ne.jp/~hoe/dv-and-movie/tipsxxx/index.html)
FluxSmooth (http://forum.doom9.org/showthread.php?t=38296)
frfun7 (http://web.archive.org/web/20080905123941/http://soulhunter.chronocrossdev.com/#002)
NoMoSmooth (http://avisynth.org.ru/docs/english/externalfilters/nomosmooth.htm)
PNLM (http://sky.geocities.jp/apechironnup/junk.html)
SVPFlow (http://forum.doom9.org/showthread.php?t=164554)
TempLinearApproximate (http://forum.doom9.org/showthread.php?t=167032)
TTempSmooth (http://bengal.missouri.edu/~kes25c/)

checkmate(c1:clip; thr:int:opt; max:int:opt; tthr2:int:opt;)
DeCrawl(c1:clip; ythresht:int:opt; ythreshs:int:opt; cthresh:int:opt; temporal:int:opt; spatial:int:opt; spatialpasses:int:opt;)
DeCross(c1:clip; ThresholdY:int:opt; Noise:int:opt; Margin:int:opt; Debug:int:opt;)
DeDot(c1:clip; luma2d:int:opt; lumaT:int:opt; chromaT1:int:opt; chromaT2:int:opt;)
DeGrainMedian(c1:clip; limitY:int:opt; limitUV:int:opt; mode:int:opt; interlaced:int:opt; norow:int:opt;)
dw3DNR(c1:clip; st:float:opt; y:int:opt; uv:int:opt; th:float:opt; sel:int:opt; debug:int:opt;)
FluxSmoothST(c1:clip; temporal_threshold:int:opt; spatial_threshold:int:opt;)
FluxSmoothT(c1:clip; temporal_threshold:int:opt;)
frfun7(c1:clip; lambda:float:opt; T:float:opt; Tuv:float:opt;)hqdn3d(c1:clip; ls:float:opt; cs:float:opt; lt:float:opt; ct:float:opt; restart:int:opt;)
NoMoSmooth(c1:clip; motion_threshold:int:opt; temporal_radius:int:opt; temporal_threshold:int:opt; spatial_radius:int:opt; spatial_threshold:int:opt; show:int:opt;)
PNLM(c1:clip; strength:int:opt; wspan:int:opt; tspan:int:opt; threads:int:opt; opt:int:opt;)
PNLM2(c1:clip; strength:int:opt; wspan:int:opt; tspan:int:opt; selfw:int:opt; angw:int:opt; threads:int:opt; opt:int:opt;)
SVAnalyse(c1:clip; s2:data; src:clip:opt;) # Needs to be added to the prefetch list.
SVConvert(c1:clip; isb:int:opt;) # Prefetch warning unknown because SVAnalyse has to called prior to this.
SVSmoothFps(c1:clip; c2:clip; c3:clip; s4:data; url:data:opt; sar:float:opt; mt:int:opt;) # Prefetch warning unknown because SVAnalyse has to called prior to this.
SVSuper(c1:clip; s2:data;) # By itself it does not display prefetch warning.
TempLinearApproximate(c1:clip; radius:int:opt; plane:int:opt;)
TTempSmooth(c1:clip; maxr:int:opt; lthresh:int:opt; cthresh:int:opt; lmdiff:int:opt; cmdiff:int:opt; strength:int:opt; scthresh:float:opt; fp:int:opt; vis_blur:int:opt;
debug:int:opt; interlaced:int:opt; pfclip:clip:opt;)
TTempSmoothF(c1:clip; maxr:int:opt; lthresh:int:opt; cthresh:int:opt; strength:int:opt; scthresh:float:opt; fp:int:opt; vis_blur:int:opt;
debug:int:opt; interlaced:int:opt; pfclip:clip:opt;)



Here's a list of the filters that currently do not work with VapourSynth.


Bifrost (http://ivtc.org/avisynth/)
TComb (http://bengal.missouri.edu/~kes25c/)

Error:
Invoke not fully implemented, tried to call: SeparateFields but I will pretend it doesn't exist
Avisynth Error: escaped IScriptEnvironment::NotFound exceptions are non-recoverable, crashing...
TComb(c1:clip; mode:int:opt; fthreshL:int:opt; fthreshC:int:opt; othreshL:int:opt; othreshC:int:opt; map:int:opt; scthresh:float:opt; debug:int:opt; opt:int:opt;)
Bifrost(c1:clip; altclip:clip:opt; scenelumathresh:float:opt; variation:int:opt; conservativemask:int:opt; interlaced:int:opt;)




MipSmooth (http://forum.doom9.org/showthread.php?p=760915#post760915)
* Freezed VDub and AVSMeter
Error:
Invoke not fully implemented, tried to call: BilinearResize but I will pretend it doesn't exist
Avisynth Error: escaped IScriptEnvironment::NotFound exceptions are non-recoverable, crashing...
MipSmooth(c1:clip; spatial:int:opt; temporal:int:opt; spatial_chroma:int:opt; temporal_chroma:int:opt; scenechange:float:opt; method:data:opt; downsizer:data:opt;
upsizer:data:opt; scalefactor:float:opt; weigh:int:opt; preset:data:opt; storecustom:int:opt; show:int:opt;)



*** The following 3 plugins only work with YUY2, and they all silently crashed VDub. I had to quickly take a screen grab of VSMeter in order to get the error message before it crashed.

DePulse (http://www5e.biglobe.ne.jp/~hoe/dv-and-movie/tipsxxx/index.html)
fGBlur (http://sky.geocities.jp/apechironnup/) - Part of the fPMD plugin.
GuavaComb (http://avisynth.org.ru/docs/english/externalfilters/guavacomb.htm) - If there's ever a fix, it also needs to be added to the prefetch list.

Error:
Invalid plane stride requested
Exception KeyError: KeyError<5376,> in <module 'threading' from 'C:\\Python33\\Lib\\threading.py'> ignored
DePulse(c1:clip; h:int:opt; l:int:opt; d:int:opt; debug:int:opt;)
fGBlur(c1:clip; sigma:float:opt;)
GuavaComb(c1:clip; Mode:data:opt; Recall:int:opt; MaxVariation:int:opt; Activation:int:opt;)




*** I couldn't get the following plugin it to work. I tried writing the parameters every way possible is VS but no luck.
I'm guessing it's because of duplicate parameter names? :)

cc (http://www.chiyoclone.net/details.html#cc)

cc(c1:clip; y1:int:opt; y2:int:opt; c1:int:opt; c2:int:opt; interlaced:int:opt; yc:float:opt; ylimit:int:opt; climit:int:opt;)



I still have a lot more plugins to test. I will then (hopefully) make a comprehensive list of all working AVISynth plugins.

One thing I'm still wondering, what are some of the benefits of adding plugins to the prefetch list?
Also, could you please explain a little about the different prefetch schemes?

#define OTHER(fname) if (name == #fname) return PrefetchInfo(1, 1, 0, 0);
#define SOURCE(fname) if (name == #fname) return PrefetchInfo(1, 1, 0, 0);
#define PREFETCHR0(fname) if (name == #fname) return PrefetchInfo(1, 1, 0, 0);
#define PREFETCHR1(fname) if (name == #fname) return PrefetchInfo(1, 1, -1, 1);
#define PREFETCHR2(fname) if (name == #fname) return PrefetchInfo(1, 1, -2, 2);
#define PREFETCHR3(fname) if (name == #fname) return PrefetchInfo(1, 1, -3, 3);
#define PREFETCH(fname, div, mul, from, to) if (name == #fname) return PrefetchInfo(div, mul, from, to);

[Myrsloik]

The OTHER and SOURCE macros don't really do anything. They're just for show and a reminder to myself about which functions I've actually checked. The PREFETCHX ones are the radius around the current frame that needs to be prefetched. For example PREFETCH0 means that a filter needs frame n as input to output frame n. PREFETCH1 means that n-1, n, n+1 are needed as input for frame n. You get the point.

The general PREFETCH macro allows you to divide and multiply the output frame number before determining which input frames are needed. So filters like decimate can be made to work.

The main reason for the prefetch stuff is that it makes implementing threading for avisynth filters a lot easier. With prefetching all frames a filter will need as input to produce a single frame will be available from the start.
If they're not available the filter will, like in single threaded avisynth, have to call the filter above it in the chain and wait for it to return. This also means that the filter instance can't be used by another thread either since avisynth filters are all single threaded. So then the other threads most likely have to sit around and wait and do nothing.

Note that avisynth-mt gets around this by spawning a lot of filter instances instead.

I hope this is clear enough.

[06_taro]

Myrsloik, could you please add "AviSynth function name: " before those prefetch and invoke warnings, so that it would be much easier to figure out which filter is not working as expected.

[Adub]

Myrsloik, kind of different question, but I figure you would know.

How often in video processing is there a operation dependence between planes? For example, is there a time when a filter needs information from the chroma planes to perform an operation on the luma plane?

The reason I ask is that I am considering adding stream support to my CUDA port. It might be incorporated into a later version, but it has some strong capabilities. Streams are a CUDA-ism that let the developer run multiple kernels in parallel on a CUDA device. This would allow for the ability to operate on all luma and chroma planes in parallel.

The only time I can think of an inter-plane dependence is possibly when performing motion compensation, so I thought I'd ask.

Simple filters like invert would see a nice speed boost, as they would likely only take as long as the luma plane to process, considering that there is much less data in the chroma planes (in subsampled video, that is).

[Myrsloik]

If you look at tbe existing avisynth fjlters it's actually quite rare. As you said motion estimation (with chroma included), ivtc and deinterlacing are the kinds that most often build a mask based on all planes and then applies it to each plane.
Even a simple filter like tweak or histogram needs it. If I have to pull a number out of my... hat I'd say about 20% of all filters need it.

[Adub]

Okay, thanks for your response, Myrsloik.

Luckily, stream-enabled kernels are optional, as CUDA issues all work to the default (0) stream anyways. I'll look at the simplest way of adding it to the core so that filter developers can use it if they want and ignore it if they don't.

It's likely just going to be some standard getCUDAStream() procedure with a pre-allocated pool of streams.

The pre-allocation is useful because I ran some tests with stream generation per frame kernel and there is enough startup overhead that it negates the parallel processing. Plus, only working with 3-32 streams is much easier to debug and profile (especially profile) than something like 2880 streams (damn near hangs NVidia's Visual Profiler).

[paradoxical]

There are other contributors to it, though, through work on core and by porting filters. It's mostly Myrsloik on the core since he knows the code the best. If you want there to be more developers, then help out.

I've been thinking of doing some work, but most of my time is spent working on my main project, bdsup2sub++, so I don't have as much free time ATM for other stuff. Once I get a new major release done for that, I'd love to see what I can pick off of Myrsloik's TODO list.

[Myrsloik]

Why aren't there any other developers besides the author?

Because the rest of the world can't see how great I am!

...or something.

[Mystery Keeper]

Personally I haven't given VapourSynth a shot yet because I couldn't install it. It wouldn't see my properly installed Python 3.

[LoRd_MuldeR]

Personally I haven't given VapourSynth a shot yet because I couldn't install it. It wouldn't see my properly installed Python 3.

I think it would be nice if the installer would give some more info, e.g. offer to download Python 3 now, instead of just aborting with a harsh error message.

Also: Wouldn't it be possible to bundle the required version of Python with Vapoursynth and create a "private" Python installation instead of messing with the "global" one?

[Myrsloik]

I think it would be nice if the installer would give some more info, e.g. offer to download Python 3, instead of just aborting with a harsh error message.

Also: Wouldn't it be possible to bundle the required version of Python with Vapoursynth and create a "private" Python installation instead of messing with the "global" one?

Python is fairly global by design. I really don't want to poke around and integrate it. The installation instructions clearly mention it as well.

[LoRd_MuldeR]

Well, I'm working on an application that makes use of Python too. I simply bundle the Python DLL and deploy it to the same folder where my "main" EXE file is located. The Python "lib" directory with all the .PY files can be compressed into a single ZIP file and put in to the same folder where the Python DLL is located. Python will look for it at the same name as the Python.DLL, only the .DLL extension replaced with a .ZIP extension. Finally, the Python modules we have to deliver as DLL (PYD) files need to be deployed as individual files. I just deploy them into a sub-folder inside my application's install folder and set the correct path at runtime, when I initialize the Python interpreter. So it's all very doable, if you want it. Just an idea though...

[Myrsloik]

Well, I'm working on an application that makes use of Python too. I simply bundle the Python DLL and deploy it to the same folder where my "main" EXE file is located. The Python "lib" directory with all the .PY files can be compressed into a single ZIP file and put in to the same folder where the Python DLL is located. Python will look for it at the same name as the Python.DLL, only the .DLL extension replaced with a .ZIP extension. Finally, the Python modules we have to deliver as DLL (PYD) files need to be deployed as individual files. I just deploy them into a sub-folder inside my application's install folder and set the correct path at runtime, when I initialize the Python interpreter. So it's all very doable, if you want it. Just an idea though...

I don't initialize the python interpreter in all cases. The vision is to make it work just like any other module and with the ability to export raw frame data to other python modules, such as numpy/whatever. I see no real reason to create a big bundle. I'm much more likely to integrate the pismo mount stuff in the installer actually.

[Mystery Keeper]

Problem fixed. Installator wouldn't find my Python because it was x64. Reinstalled x32, and it installed alright.

[Adub]

Myrsloik, I have a design question for you.

I'm at the point where I have a pretty stable CUDA implementation in the core, as well as a helper plugin for shipping frames back and forth on the GPU.

At this point, I'd like to start porting over the simplefilters that actually do processing in GetFrame() (Lut/Lut2/Transpose, etc). However, I'm a little unsure as to the best away to do this, as CUDA code is generally encapsulated in a separate ".cu" file, which would separate it from the current static nature of the simplefilters.c.

I did think about creating a separate plugin that is CUDA only, but that would be a crap ton of code duplication for all of the other functions (Create,Init,Free) for each filter in simplefilters.

I wanted to get your input on this before I start banging anything out.

For reference, I've been adding some work to the core, and wrapping anything that is CUDA/GPU specific in a "#if FEATURE_CUDA" block if it is in the .h or .cpp file. All other functionality is added to the new "vscore.cu" file.

Edit: Thinking about this further, if there was a way I could force Waf to only use NVCC to compile simplefilters.c instead of gcc, I could write the code directly into simplefilters.c I think using preprocessor checks.

[Adub]

Myrsloik, I have a design question for you.

I'm at the point where I have a pretty stable CUDA implementation in the core, as well as a helper plugin for shipping frames back and forth on the GPU.

At this point, I'd like to start porting over the simplefilters that actually do processing in GetFrame() (Lut/Lut2/Transpose, etc). However, I'm a little unsure as to the best away to do this, as CUDA code is generally encapsulated in a separate ".cu" file, which would separate it from the current static nature of the simplefilters.c.

I did think about creating a separate plugin that is CUDA only, but that would be a crap ton of code duplication for all of the other functions (Create,Init,Free) for each filter in simplefilters.

I wanted to get your input on this before I start banging anything out.

For reference, I've been adding some work to the core, and wrapping anything that is CUDA/GPU specific in a "#if FEATURE_CUDA" block if it is in the .h or .cpp file. All other functionality is added to the new "vscore.cu" file.

Edit: Thinking about this further, if there was a way I could force Waf to only use NVCC to compile simplefilters.c instead of gcc, I could write the code directly into simplefilters.c I think using preprocessor checks.

Okay, I think i've figured out a way I can do this. Essentially, I'll create a simpefilters.cu file that holds all of the GPU kernels and any associated wrappers that are necessary for calling them. Then, in simplefilters.c, I'll implement a small #if-surrounded block that will allow for the use of these new kernels if Vapoursynth is compiled with CUDA support.

I think I can even get it so that we can have the GPU and CPU filtering capabilities live side-by-side, and just filter the frame based off of it's FrameLocation. That way there is little extra work that needs to be done by the user.

Edit: Er, didn't mean to make a new post, don't know how that happened.

[Selur]

btw. the starter script over at http://www.vapoursynth.com/doc/gettingstarted.html
import vapoursynth as vs
# needed for stdout
import sys
# create a core instance
core = vs.Core()
# load a native vapoursynth plugin
# you should use absolute paths as the working directory may not be what you think it is
core.std.LoadPlugin(path=r'c:\plugins\ffms2.dll')
# load an avisynth plugin
# the loaded functions will always end up in the avs namespace
core.avs.LoadPlugin(path=r'c:\avisynth\UnDot.dll')
# open a video file; ret is now a clip object
ret = core.ffms2.Source(source='Super Size Me.avi')
# apply the undot filter to the video
ret = core.avs.UnDot(clip=ret)
# output the clip to stdout with y4m headers (useful for x264 encoding/mplayer playback)
ret.output(sys.stdout, y4m=True)
has to mistakes:

ret = core.ffms2.Source(source='Super Size Me.avi') need to be:
ret = core.ffms2.Source(source=r'Super Size Me.avi')
ret = core.avs.UnDot(clip=ret) needs to be:
ret = core.avs.UnDot(ret)

if the intention of the page is not to scare away users it might be a good idea to fix this. :D

Cu Selur

[jackoneill]

btw. the starter script over at http://www.vapoursynth.com/doc/gettingstarted.html

has to mistakes:

ret = core.ffms2.Source(source='Super Size Me.avi') need to be:
ret = core.ffms2.Source(source=r'Super Size Me.avi')
ret = core.avs.UnDot(clip=ret) needs to be:
ret = core.avs.UnDot(ret)

if the intention of the page is not to scare away users it might be a good idea to fix this. :D

Cu Selur
I'm not sure about the second one, but the first one is not a mistake. You don't /have to/ use raw strings. It's just more convenient when you have backslashes in your paths (like in Windows).

[Chikuzen]

#!/usr/bin/env python3

import vapoursynth as vs
import sys

core = vs.Core()
std = core.std

def ivtc(clip, offsets, cycle=10, tff=True):
clip = std.SeparateFields(clip, tff)
clip = std.DoubleWeave(clip, tff)
return std.SelectEvery(clip, cycle, offsets)

def show_progress(c, t):
print('%i / %i' % (c, t), end='\r', file=sys.stderr, flush=True)

src = std.BlankClip(width=1920, height=1080, format=vs.YUV420P8, length=50000)

clips = []
for f in range(200):
clips.append(ivtc(src[f * 100: f * 100 + 30], [1, 4, 6, 9]))
clips.append(ivtc(src[f * 100 + 30: f * 100 + 60], [0, 2, 5, 7]))
clips.append(ivtc(src[f * 100 + 60: f * 100 + 100], [1, 3, 6, 8]))

last = std.Splice(clips)
#print(last, file=sys.stderr)
last.output(sys.stdout, False, 0, show_progress)


When I ran this script, the speed falls at about 6000th frame and the amount of the memory used continued increasing.
https://dl.dropbox.com/u/19797864/isthismemleak.jpg
Is this bug or limitation ?

[Adub]

I can confirm this behaviour on Ubuntu 12.04, 64-bit, Python version 3.2.3.

The speed definitely slows down, but I'm not sure if it's a constant decrease. However, my memory usage did spike as high as 3.2GB.

[Reel.Deel]

I couldn't help myself, I ran Chikuzen's script using VFW and got similar results.
I also used AVSMeter to get a nice log with CPU usage and speed. Unfortunately it crashed when the memory got around 1650mb.
Regardless, here's the log (https://www.dropbox.com/s/jrua9wo6ir616oe/VS%20Test.log).

[Myrsloik]

I think I know at least half the issue. There is a missing free of the underlying frame data reported by ADub that's fixed in svn. I think that's what causes you to run out of memory after thousands of frames. There may be another issue with the cache strategy as I haven't tested it on bigger/complicated scripts. Guess I'll prepare a new test version later today so you can see if it really did get fixed.

[Adub]

Actually Myrsloik, my confirmation two posts above was with the latest SVN changes (rev 442 last time I checked). I'm not quite sure where the leak is at this point.

[gnaggnoyil]

Is there anyway to "fade", or to "dissolve" some frames of a clip in vapoursynth? In avisynth I can use the "fade" and the "dissolve" function. Is it able to do so in vapoursynth? I can't find any documents about it in the offical website, nor can I find it in doom9 :(

[jackoneill]

Is there anyway to "fade", or to "dissolve" some frames of a clip in vapoursynth? In avisynth I can use the "fade" and the "dissolve" function. Is it able to do so in vapoursynth? I can't find any documents about it in the offical website, nor can I find it in doom9 :(

Assuming "fade" in avisynth does what I think it does:

fade_input = # your source here
blank = core.std.BlankClip(clip=fade_input, length=1, color=[255, 128, 128]) # assumes you're working with 8 bit YUV
fade_frames = []
for i in range(fade_input.num_frames):
fade_frames.append(core.std.Merge(clips=[fade_input[i], blank], weight=i/(fade_input.num_frames-1)))
fade_output = core.std.Splice(clips=fade_frames)


Tweak the colour of the blank clip to change the colour it fades to/from.
Tweak the parameters of range() to make it fade /in/, rather than out (make it count backwards).
Replace blank with some other clip, say fade_input2[i], to make it fade from one scene to another.

~~~~~

I don't know what "dissolve" should look like.

[gnaggnoyil]

thanks a lot :) it does helps. I didn't know that videoNode[i] returns a one-frame clip including the i-th frame of videoNode before, now I got it :) Really much more powerful than avisynth.

[digitall.h]

Hi @all,
is it happening further development of Vapoursynth in the background?.
I read in Vapoursynth project page that Myrsloik had a new job (sort of?) and development would slow down. That was published in the beginning of March. Was just Myrsloik developing Vapoursynth?.

It would be sad, since this project looks really promising, specially for those of us living in the Linux world, and missing avisynth natively.

I wish I could be of a help, but I'm in the "Hello world!" phase of Python...:(
I just can help testing.

[Myrsloik]

I've only written a few more lines of code since the last post. It's not dead but I spend all day programming other things... for money. It's actually not that much more code needed to make the core part complete so it'll definitely happen some day.

Also, real life takes time.

[digitall.h]

I've only written a few more lines of code since the last post. It's not dead but I spend all day programming other things... for money. It's actually not that much more code needed to make the core part complete so it'll definitely happen some day.
Great!, it's good to know this.

Also, real life takes time.
Of course.
I really admire, respect and thank the time programmers like you and many others in this site spend creating projects and sharing them with everybody.

:)

[Adub]

@digitall.h
Is there something missing from Vapoursynth that you'd like to see? I ask because you speak of the project being promising, which just sounds to me personally like it's missing something you want.

And I'm totally with you on the Linux support. I do all of my Vapoursynth development and usage on Linux.

[kolak]

Yes:
-native QTGMC or at least yadifmod
-better "support" for interlaced video
-better source filters (ffms2, avisource still have problems)
-8bit+ support for core filters- like deinterlacing, etc
-frame prefetching support for avs filters
-many more

Example of great vapoursynth filter implementation is fmtconv from Cretindesalpes.

[sl1pkn07]

in my knowledge: (a mode of information)

cross plataform plugins:

convo2d
d2vsource
ffms2
fmtconv
genericfilters
histogram
lsmashsource
nnedi3
scenechange
scxvid
temporalsoften
vsimagereader
vsrawsource

windows plugins:

VS_AvsReader (work with avisynth plugins)
vscolormatrix (?) (don't build on my linux)

mixed plugins (use avisynth plugins to work)

havsfunc

[Adub]

I suppose I should have been a bit more clear, as I ment develop to the actual Vapoursynth project.

Almost everything that you have listed can be done in a plugin or script format for Vapoursynth's use, which can be done by any willing developer, not just Myrsloik.

The biggest hurdle for QTGMC at this point is the MC part. I know that Myrsloik wanted to look into a new version of MVTools for Vapoursynth, but that is a large task.

Currently all of the core filters support 8/9/10/16 bit video (with some even supporting float). But I agree, lack of a proper deinterlacer is a bit of a problem.

As for the "better source filters", what problems do you have with FFMS2? I've used it in numerous tests before and I've had no problems.

[kolak]

Except fmtconv and some source filters I don't think anything else supports 8bit+.

Avisource has speed problem (not a big deal for now), ffms2- well it sort of works, but far from being reliable. There is no official build, some random builds etc. I think vapoursynth needs reliable and well written source filter, as without it rest of the engine can't be used properly.

I know Myrsloik is busy, which is shame, but he needs real life and money :)

[06_taro]

Myrsloik, I met some strange results from Lut and Expr for YUV420P16.

I used a simple script to reproduce it:
import vapoursynth as vs
core = vs.Core()
std = core.std
avs = core.avs
avs.LoadPlugin('DGDecodeNV.dll')
dgi = 'test.dgi'
clip = avs.DGSource(dgi)
clip = core.fmtc.bitdepth(clip, bits=16)

lut = []
for x in range(2**clip.format.bits_per_sample):
lut.append( x )

res = std.Lut(clip, lut, [0, 1, 2])


or I changed the last line to:
res = std.Expr( clip, 'x' )

It is expected that Lut / Expr should return exactly the original value of pixel, and is same as clip. However, with Lut, I got a nearly blank (0, 0, 0) msb, and few information in lsb related to the original clip; while with Expr, the whole picture turned pink, and looked as if it was the input clip horizontally interleaved a full (65535, 65535, 65535) clip.

Some other filters which support YUV420P16 like generic.Blur gives expected results, so I suppose something went wrong in Lut and Expr, or in my script. Naturally, Expr(clip, '') works as it should, so does specifying planes[] to preserve data in certain planes. Using a const in expression like lut.append(32768) or Expr(clip, '32768') also works well. Did I make any mistakes, or do these two functions need special workaround like re-order bytes, or is there any regression in the filters?

Env:
Win Server 2012 Datacenter (x64)
python 3.3 win32
vapoursynth r18
Any video, any source filter, any resizer for 8->16 conversion, and any other pre-filters, as long as the input clip for Lut/Expr is YUV420P16.
Viewed by vfw dithered by fmtc.bitdepth, or encoded by FFmpeg/x264 with y4m or raw with specified format/input-depth/csp, or raw yuv generated by clip.output() function.

Didn't test other csps, as I'm still using masktools for 8-bit stuffs :P

[Adub]

Hmm, good spot, 06_taro.

Just looking at the code, with no testing done what so ever, it may be due to the fact that the srcp pointer is not being cast to uint16_t when being read in LUT.

Ex:
((uint16_t *)dstp)[x] = lut[srcp[x]];
should be
((uint16_t *)dstp)[x] = lut[((const uint16_t *)srcp)[x]];

As for Expr, the C-code looks fine, but the plugin normally runs with the ASM code, which I don't read well/at all. I assume something similar may be happening.

It might be good to add a regression test in the Python tests, just for good maintenance sake. Hell, I can write one when I get home.

Edit: Looks like the same issue exists in LUT2:
((uint16_t *)dstp)[x] = lut[(srcpy[x] << shift) + srcpx[x]];

[Adub]

Here's a patch to simplefilters.c which should fix LUT and LUT2

diff --git a/src/core/simplefilters.c b/src/core/simplefilters.c
index 9ea2a97..738c337 100644
--- a/src/core/simplefilters.c
+++ b/src/core/simplefilters.c
@@ -1885,7 +1885,7 @@ static const VSFrameRef *VS_CC lutGetframe(int n, int activationReason, void **i

for (hl = 0; hl < h; hl++) {
for (x = 0; x < w; x++)
- ((uint16_t *)dstp)[x] = lut[srcp[x]];
+ ((uint16_t *)dstp)[x] = lut[((uint16_t *)srcp)[x]];

dstp += dst_stride;
srcp += src_stride;
@@ -2052,7 +2052,7 @@ static const VSFrameRef *VS_CC lut2Getframe(int n, int activationReason, void **

for (hl = 0; hl < h; hl++) {
for (x = 0; x < w; x++)
- ((uint16_t *)dstp)[x] = lut[(srcpy[x] << shift) + srcpx[x]];
+ ((uint16_t *)dstp)[x] = lut[(((uint16_t *)srcpy)[x] << shift) + ((uint16_t *)srcpx)[x]];

dstp += dst_stride;
srcpx += src_stride;

And a new regression test framework under test/test_regression.py
import unittest
import vapoursynth as vs

class RegressionTestSequence(unittest.TestCase):

def setUp(self):
self.core = vs.Core()

def checkDifference(self, cpu, gpu):
diff = self.core.std.PlaneDifference([cpu, gpu], 0, prop="PlaneDifference0")
diff = self.core.std.PlaneDifference([diff, gpu], 1, prop="PlaneDifference1")
diff = self.core.std.PlaneDifference([diff, gpu], 2, prop="PlaneDifference2")

for i in range(diff.num_frames):
frame = diff.get_frame(i)
self.assertEqual(frame.props.PlaneDifference0[0], 0)
self.assertEqual(frame.props.PlaneDifference1[0], 0)
self.assertEqual(frame.props.PlaneDifference2[0], 0)

def testLUT16Bit(self):
clip = self.core.std.BlankClip(format=vs.YUV420P16, color=[69, 242, 115])

lut = []
for x in range(2 ** clip.format.bits_per_sample):
lut.append(x)

ret = self.core.std.Lut(clip, lut, [0, 1, 2])

self.checkDifference(clip, ret)

if __name__ == '__main__':
unittest.main()

[sl1pkn07]

one little fix

https://code.google.com/p/vapoursynth/source/browse/trunk/include/VSScript.h#21

is "#include VapourSynth.h" (#include "vapoursynth.h" fail build in linux (case sensitive))

[Selur]

Here's what I tried to get Vapoursynth working in my Linux Mint 64bit 13.04 (inside a Virtualbox) working:

1. Installing a bunch of dependencies.
sudo apt-get install build-essential libqtcore4 python3 python3-dev wget subversion yasm libswscale-dev libavcodec-dev libass-dev autoconf libavformat-dev mplayer
2. Extracting and installing Cython (http://pypi.python.org/packages/source/C/Cython/Cython-0.19.zip):
sudo python3 setup.py install
3. Checkout, build and install Vapoursynth:
3.1 checkout
svn checkout http://vapoursynth.googlecode.com/svn/trunk vapoursynth
cd vapoursynth
3.2 fixing typo in include/VSScript.h:
[code]#include "vapoursynth.h"
to#include "VapourSynth.h"
3.3 installing
./bootstrap.py && ./waf configure && ./waf build && sudo ./waf install
sudo ./setup.py install
sudo ldconfig
4. checking out, extracting and building latest libav (https://libav.org/releases/libav-9.5.tar.gz), since Ubuntu 13.04 is missing libavresample
5. checking out and building ffms2
svn checkout http://ffmpegsource.googlecode.com/svn/trunk ffms2
cd ffms2
./autogen.sh
./configure --enable-shared
make

5. creating an example script named vstestscrpt.vpy:
#!/usr/bin/python3
import vapoursynth as vs
import sys
core = vs.Core()
core.std.LoadPlugin("/usr/local/lib/libffms2.so")
ret = core.ffms2.Source("/home/selur/Desktop/test.avi")
ret.output(sys.stdout, y4m=True)
6. making the script excecuteable
chmod +x vstestscrpt.vpy
7. calling mplayer to test the whole thing:
$ ./vstestscript.vpy | mplayer

Problem is the last step gives me:
Traceback (most recent call last):
File "./vstestscript.vpy", line 7, in <module>
ret.output(sys.stdout, y4m=True)
File "vapoursynth.pyx", line 676, in vapoursynth.VideoNode.output (build/src/cython/vapoursynth.c:10747)
vapoursynth.Error: 'write() call returned error'


-> Does anyone have an idea what is doing wrong? (or is this a bug?)

Cu Selur

[sneaker_ger]

Is mplayer set to y4m input via pipe? Because that's what you've set your VapourSynth output to.

[Selur]

doh, I was missing a '-', needs to be 'mplayer -'

[sl1pkn07]

for Archlinux users: https://aur.archlinux.org/packages/?O=0&K=vapoursynth

one little question, cpython is dependency for run or only for build?

greetings

[gnaggnoyil]

can i use avisynth runtime functions in a vapoursynth script? i.e., can i let core.avs.IsCombedTIVTC() work in vapoursynth, either using it in core.std.SelectClip(), or just return a boolean list, or do anything else?
if can't, what else could be used to replace IsCombedTIVTC for above usage?

[Adub]

@Selur,

Just a heads up, Mplayer/MPV might prefer that you disable the "y4m" setting on output, as setting it to true outputs an extended Y4M format that requires specific support by the reader, such as X264. Setting y4m=False still outputs standard y4m, and should be quite compatible with any command line player.

[qyot27]

@Selur,

Just a heads up, Mplayer/MPV might prefer that you disable the "y4m" setting on output, as setting it to true outputs an extended Y4M format that requires specific support by the reader, such as X264. Setting y4m=False still outputs standard y4m, and should be quite compatible with any command line player.
So long as mplayer* or mpv were actually built against FFmpeg and not the fork, the extended formats should work fine. At least the first time, maybe (I still haven't figured that part out; I think something's going wonky in Python or Cython so that subsequent attempts don't work). Alternately, always make sure to thread it through ffmpeg first to standardize the headers. It's always do-able that way.
./script.vpy | ffmpeg -i - -f yuv4mpegpipe -strict -1 - | mpv -


*for mplayer there's no guarantee that the repository version was (well, referring to mplayer2 here; mplayer-svn's situation is out of my realm of experience, but I think it uses an internal copy of FFmpeg unless there's some way of forcing that off); there is no mpv build in the repositories yet, so unless there's some PPA floating around, the user is in total control over which one they build into it (as they also are if they decide to build mplayer from source, but personally I see no reason not to use mpv if that's what you're going to do anyway).

[Myrsloik]

Here's a patch to simplefilters.c which should fix LUT and LUT2


It's not quite right for lut2. Consider the case where clip1 is 8bit and clip2 is 16bit and you'll see why. I applied the fix for lut and added the testcase.

If you want to compile from svn then I I suggest you revert r444-445 if you see any issues. That's the unfinished stuff.

[06_taro]

Thanks Adub & Myrsloik for the fix! Not really eager to get a fixed 16-bit LUT2, since even 8bit + 16bit LUT2 with 16bit output seems to need 2^8*2^16*2=32MB table, which is still too large for my CPU cache.

Any further idea about the regression in Expr?

[Adub]

It's not quite right for lut2. Consider the case where clip1 is 8bit and clip2 is 16bit and you'll see why. I applied the fix for lut and added the testcase.

If you want to compile from svn then I I suggest you revert r444-445 if you see any issues. That's the unfinished stuff.

Ahh, I see your point then. Okay, well good to see that LUT got fixed!

@qyot27
Ahh, yeah my mpv is built against libav I believe (Ubuntu default these days), so I ran into strange issues when using the extended format. Granted, this was a few months ago, so things might have cleared up since then.

[qyot27]

@qyot27
Ahh, yeah my mpv is built against libav I believe (Ubuntu default these days), so I ran into strange issues when using the extended format. Granted, this was a few months ago, so things might have cleared up since then.
It most probably won't have cleared up, since a search of libav's git repo shows absolutely nothing related to the extended format. It looks like they still don't support >10 <16 bit YUV (so those 12- and 14-bit H.264 samples wouldn't be treated correctly either).

If only it weren't intrinsically tied to VFW (which would sort of defeat the purpose of the name) and 2.6's colorspaces, it'd be interesting if the HBVFWSource plugin could be ported to AvxSynth, as that would at least avoid having to pipe into either x264 or FFmpeg (or mplayer/mpv by extension). Not really a long-term solution, but possibly less of a hassle in the short term until a discrete demuxer for VapourSynth coalesces in one or both of them.

[Myrsloik]

Ahh, I see your point then. Okay, well good to see that LUT got fixed!

@qyot27
Ahh, yeah my mpv is built against libav I believe (Ubuntu default these days), so I ran into strange issues when using the extended format. Granted, this was a few months ago, so things might have cleared up since then.

I didn't fix lut2 though. I still have far too much other programming to do.

[Adub]

Hmm, if I get some free time from writing my thesis, I'll see if I can come up with a solution for LUT2. No guarantees on that one though.

The mixed use case of 8-bit and 16-bit inputs is a bit of a problem.

[digitall.h]

@digitall.h
Is there something missing from Vapoursynth that you'd like to see? I ask because you speak of the project being promising, which just sounds to me personally like it's missing something you want.
...

Adub, sorry me for the delay in my answer, that probably doesn't make sense at this point of the discussion... :scared:

As I said no programming skills here (beginning with python, just 'Hello world!', just a hobby). I stated I find it promising for us end users, waiting for the point when Vapoursynth becomes more user friendly.

I mean, standard Vapoursynth 'scripts' to carry out those filter chains we used with avisynth under Windows. Some were already suggested in previous posts. As an example, the last avisynth scripts I was using was:
LoadPlugin("C:\Filtro25\DGDecode.dll")
LoadPlugin("C:\Filtro25\RemoveGrainSSE2.dll")
LoadPlugin("C:\Filtro25\SSE2Tools.dll")
LoadPlugin("C:\Filtro25\RepairSSE2.dll")
LoadPlugin("C:\Filtro25\mt_masktools.dll")
Import("C:\Filtro25\LRDLSFs.avsi")
Import("C:\Filtro25\Soothe.avsi")
Mpeg2Source("C:\test.d2v",idct=4)
Crop(0,74,720,428,true)
LRemoveDust_YV12(17,2)
dull = last.BilinearResize(672,400)
sharp = last.LimitedSharpenFaster(dest_x=672,dest_y=400)
Soothe(sharp,dull,18)
Addborders(16,88,16,88)

Where function Soothe.avsi was:
function Soothe(clip sharp, clip orig, int "keep")
{
keep = default(keep, 24)
keep = (keep>100) ? 100 : (keep<0) ? 0 : keep
KP = string(keep)
diff = mt_lutxy(orig,sharp,"x y - 128 +", U=1,V=1)
diff2 = diff.temporalsoften(1,255,255,32,2)
diff3 = mt_lutxy(diff,diff2, "x 128 - y 128 - * 0 < x 128 - 100 / " + KP
\ + " * 128 + x 128 - abs y 128 - abs > x " + KP
\ + " * y 100 " + KP + " - * + 100 / x ? ?", U=1,V=1)
return( mt_lutxy(orig,diff3,"x y 128 - -",U=2,V=2) )
}
And function LRDLSFs.avsi was:
function LRemoveDust_YV12(clip input, int clmode, int "limit")
{
limit=default(limit,2)
clmode=default(clmode,17)
repmode = 2
clensed = Clense(input)
rep = Repair(clensed, input, mode=repmode)
rg = RemoveGrain(rep, mode=clmode)
return LimitChange(rg, input, limit)
}


function LimitedSharpenFaster( clip clp,
\ float "ss_x", float "ss_y",
\ int "dest_x", int "dest_y",
\ int "Smode" , int "strength", int "radius",
\ int "Lmode", bool "wide", int "overshoot", int "undershoot",
\ int "soft", int "edgemode", bool "special",
\ int "exborder" )
{
ox = clp.width
oy = clp.height

Smode = default( Smode, 3 )
ss_x = (Smode==4)
\ ? default( ss_x, 1.25)
\ : default( ss_x, 1.5 )
ss_y = (Smode==4)
\ ? default( ss_y, 1.25)
\ : default( ss_y, 1.5 )
dest_x = default( dest_x, ox )
dest_y = default( dest_y, oy )
strength = (Smode==1)
\ ? default( strength, 160 )
\ : default( strength, 100 )
strength = (Smode==2&&strength>100) ? 100 : strength
radius = default( radius, 2 )
Lmode = default( Lmode, 1 )
wide = default( wide, false )
overshoot = default( overshoot, 1)
undershoot= default( undershoot, overshoot)
softdec = default( soft, 0 )
soft = softdec!=-1 ? softdec : sqrt( (((ss_x+ss_y)/2.0-1.0)*100.0) ) * 10
soft = soft>100 ? 100 : soft
edgemode = default( edgemode, 0 )
special = default( special, false )
exborder = default( exborder, 0)
#radius = round( radius*(ss_x+ss_y)/2) # If it's you, Mug Funky - feel free to activate it again
xxs=round(ox*ss_x/8)*8
yys=round(oy*ss_y/8)*8
smx=exborder==0?dest_x:round(dest_x/Exborder/4)*4
smy=exborder==0?dest_y:round(dest_y/Exborder/4)*4

clp.isYV12() ? clp : clp.converttoyv12()

ss_x != 1.0 || ss_y != 1.0 ? last.spline36resize(xxs,yys) : last
tmp = last

edge = mt_logic( tmp.mt_edge(thY1=0,thY2=255,"8 16 8 0 0 0 -8 -16 -8 4")
\ ,tmp.mt_edge(thY1=0,thY2=255,"8 0 -8 16 0 -16 8 0 -8 4")
\ ,"max") .mt_lut("x 128 / 0.86 ^ 255 *")

tmpsoft = tmp.removegrain(11,-1)
dark_limit1 = tmp.mt_inpand()
bright_limit1 = tmp.mt_expand()
dark_limit = (wide==false) ? dark_limit1 : dark_limit1 .removegrain(20,-1).mt_inpand()
bright_limit = (wide==false) ? bright_limit1 : bright_limit1.removegrain(20,-1).mt_expand()
minmaxavg = special==false
\ ? mt_average(dark_limit1, bright_limit1)
\ : mt_merge(dark_limit,bright_limit,tmp.removegrain(11,-1),Y=3,U=-128,V=-128)

Str=string(float(strength)/100.0)
normsharp = Smode==1 ? unsharpmask(strength,radius,0)
\ : Smode==2 ? sharpen(float(strength)/100.0)
\ : Smode==3 ? mt_lutxy(tmp,minmaxavg,yexpr="x x y - "+Str+" * +")
\ : mt_lutxy(tmp,tmpsoft,"x y == x x x y - abs 16 / 1 2 / ^ 16 * "+Str+
\ " * x y - 2 ^ x y - 2 ^ "+Str+" 100 * 25 / + / * x y - x y - abs / * + ?")

OS = string(overshoot)
US = string(undershoot)
mt_lutxy( bright_limit, normsharp, yexpr="y x "+OS+" + < y x y x - "+OS+" - 1 2 / ^ + "+OS+" + ?")
mt_lutxy( dark_limit, last, yexpr="y x "+US+" - > y x x y - "+US+" - 1 2 / ^ - "+US+" - ?")

Lmode==1 ? mt_clamp(normsharp, bright_limit, dark_limit, overshoot, undershoot) : last

normal = last
zero = mt_clamp(normsharp, bright_limit, dark_limit, 0,0)

Lmode==3 ? mt_merge(normal,zero,edge.mt_inflate()) : normal

edgemode==0 ? last
\ : edgemode==1 ? mt_merge(tmp,last,edge.mt_inflate().mt_inflate().removegrain(11,-1),Y=3,U=1,V=1)
\ : mt_merge(last,tmp,edge.mt_inflate().mt_inflate().removegrain(11,-1),Y=3,U=1,V=1)

AMNT = string(soft)
AMNT2 = string(100-soft)
sharpdiff=mt_makediff(tmp,last)
sharpdiff2=mt_lutxy(sharpdiff,sharpdiff.removegrain(19,-1),
\ "x 128 - abs y 128 - abs > y "+AMNT+" * x "+AMNT2+" * + 100 / x ?")

soft==0 ? last : mt_makediff(tmp,sharpdiff2)

(ss_x != 1.0 || ss_y != 1.0)
\ || (dest_x != ox || dest_y != oy) ? spline36resize(dest_x,dest_y) : last

ex=blankclip(last,width=smx,height=smy,color=$FFFFFF).addborders(2,2,2,2).coloryuv(levels="TV->PC")
\.blur(1.3).mt_inpand().blur(1.3).spline36resize(dest_x,dest_y,1.0,.0)
tmp = clp.spline36resize(dest_x,dest_y)

clp.isYV12() ? ( exborder==0 ? tmp.mergeluma(last)
\ : mt_merge(tmp,last,ex,Y=3,U=1,V=1) )
\ : ( exborder==0 ? tmp.mergeluma(last.converttoyuy2())
\ : tmp.mergeluma( mt_merge(tmp.converttoyv12(),last,ex,Y=3,U=1,V=1)
\ .converttoyuy2()) )

(edgemode!= -1) ? last : edge.spline36resize(dest_x,dest_y).greyscale

return last
}

Of course I did not create these scripts, where created by skilled people in Doom9. I don't know if at this point this filter chain (or something equivalent) can be translated into Vapoursynth syntax, and make it understandable for us end-users.
:)

[kolak]

Hmm, if I get some free time from writing my thesis, I'll see if I can come up with a solution for LUT2. No guarantees on that one though.

The mixed use case of 8-bit and 16-bit inputs is a bit of a problem.

Can't you convert all source to one format (regardless source precision)? It would be easier than.

[IanB]

LUT's are a fast solution to complex problems, but you must be mindful of the size of the table.

Two 16-bit inputs for a 16-bit output requires a table of 2*65536*65536 = 8,589,934,592 bytes (8GB), clearly that's not going to work in a 32 bit address space. Even in x64 there is the start up time to calculate 4 billion table entries and the slow access to such a huge table, contrast a 1920x1080 frame has only 2,073,600 pixels so raw mode you only do 1/2000 the number of calculations. Horse for courses.

Now an 8-bit input across a 16-bit input for a 16-bit output requires a table of 2*256*65536 = 33,554,432 bytes (32MB), gee that even fit's in the L3-cache of a top end machine.

And if you know your two 16-bit inputs are actually only 10-bit, then you can sneak by with a table of 2*1024*1024 = 2,097,152 (2MB) and that fits in the L2-cache of many machines.

[kolak]

Not sure how it relates to this case by in pro software LUTs are simplified and use only eg 4096 entries, like LOG to 709 LUT. Is this the same thing?

[IanB]

There are 100's of algorithms to do partial LUT acceleration. That is where the LUT only contains a small fraction of the total lookup space.

One clever algorithm I have seen had "values" and "1st derivative values" in a pair of sparse LUT's. You used the 2 LUT values to calculate quadratic approximation, ax^2+bx+c, in the local region between 2 LUT points.

[Mug Funky]

i believe most colour grading software uses that approach - you define a 16x16x16 colour cube and the GPU does the work. this gives you better than realtime processing of very big frames in 32 bit float (values are often only 10 bit because they're based off densitometer measurements of printed film compared against 10-bit scans of the print's original neg counterpart).

[Adub]

Question:

For Lut2, do we assume that clips must always be passed in in order of their bits-per-sample?

Ex:
clips=[cilpP9, clipP10],
not
clips=[clipsP10, clipsP9]

I ask because I'm looking at fixing Lut2 and judging by the example below, I see issues with the [10-bit, 9-bit] input:

lut = []
for y in range(2**clipx.format.bits_per_sample):
for x in range(2**clipy.format.bits_per_sample):
lut.append(x)
Lut2(clips=[clipx, clipy], lut=lut, planes=[0, 1, 2])


The above example with stunted values for a 10-bit output clip since the LUT never goes above 2^9 = 512 values.

However, if we assume that clipx must ALWAYS be the lesser bit clip, then it should work just fine.

If this is the case, I think it would be good to make a note in the documentation.

Edit:

I suppose a second question is format the output clip format. If we pass in an 8-bit and a 10-bit clip, then do we want an 8-bit output clip or a 10-bit output clip?
I'm thinking the latter at this point, but Lut2's current behaviour is to output an 8-bit clip (assuming it's the first clip passed in).

[Adub]

Hmm, after looking at Lut2's code for a while, I came to a few conclusions.

The current Lut2 code does not properly handle > 8-bit inputs, which we already knew. But, even more so, the example script in the documentation is a bit misleading.

The reason being that Lut2 uses the first clip's values for the column, or x, lookup in the LUT. It uses the second clip's values for the row, or y, lookup in the LUT.

The example provided in the documentation (see url) is slightly misleading.: http://vapoursynth.com/doc/functions/lut2.html

Repeated here for brevity:

lut = []
for y in range(2**clipx.format.bits_per_sample):
for x in range(2**clipy.format.bits_per_sample):
lut.append((x + y)//2)
Lut2(clips=[clipx, clipy], lut=lut, planes=[0, 1, 2])


I have highlighted a few key parts of the script.

The clips themselves don't really matter, they could be called clipA and clipB for all I care, but the manner in which they generate the LUT is import.

Specifically, in the above example clipy generates the LUT's x values in the inner loop, and clipx generates the LUT's y values in the outer loop. If the clips are the same bits per sample, that's okay, but otherwise issues can occur.

Now here's one more problem. The script passes the clips to Lut2 as [clipx, clipy]. According to the code, that would be correct if clipx corresponded to the x values of the LUT, and clipy the y values. Only, as I described above, they are actually flipped.

Now, I have a patch and associated regression tests that correctly fixes Lut2 and it's handling of different bit-depth clips, including > 8-bits. However, I'd like to clear up my questions in the previous post before I submit it, as it does make a difference which inputs are legal and the desired clip output format.

[06_taro]

I think it is reasonable to add an argument to indicates output bit depth such as 'bits' or 'format'. It is useful to tell Lut which integer type to be used to store output value. If we limits the output bit depth to input depth, or even highest input depth in Lut2, it is another pain to use some wrappers like the method Dither_lut8 uses to get higher depth output. Though with this argument one need to be careful to handle the values when building lookup table, I don't think it is a problem for users who know what lut is.

To make things easier, maybe we can make a limitation that output 'bits' must be either 8, 16, or the highest input bit depth, as 9~15 bit depth of output value won't save any space of memory compared to 16 bits, and we won't run into the mess if the output format isn't natively supported by VS. I'm even thinking of supporting only 8 or 16 bits output, so that the logic is clearer, and it is already fast enough to do rounding in fmtconv. But in this way we might loss convenience when dealing with simple calculation like Lut2([clipXp10, clipYp10], lut=simple_lut) in which simple_lut is something like (x-y).

[gnaggnoyil]

Another memory leak problem:
vapoursynth seems to have a rapid memory request when a script request every frame. for example, the script below:

import vapoursynth
import sys
core=vapoursynth.Core()
video1=SOMECLIP;
video2=SOMECLIP
video3=core.std.PlaneDifference([video1,video2],0,prop="YPlaneDifference")
l=[]
for i in range(video3.num_frames):
l.append(video3.get_frame(i).props.YPlaneDifference[0])

and this kind of script using vapoursynth r18 would result a rapid memory request in my win7 x64 system and finally cause an avisynth crash error. I guess that vapoursynth would allocate a piece of memory to save the video frame raw data each time when get_frame() is called, but I am very puzzled that how videoNode.output() method wouldn't cause such a crash if the above statement is true. Is there anyone who can point out the reason, or give a solution to this problem?

[Myrsloik]

Another memory leak problem:
vapoursynth seems to have a rapid memory request when a script request every frame. for example, the script below:

import vapoursynth
import sys
core=vapoursynth.Core()
video1=SOMECLIP;
video2=SOMECLIP
video3=core.std.PlaneDifference([video1,video2],0,prop="YPlaneDifference")
l=[]
for i in range(video3.num_frames):
l.append(video3.get_frame(i).props.YPlaneDifference[0])

and this kind of script using vapoursynth r18 would result a rapid memory request in my win7 x64 system and finally cause an avisynth crash error. I guess that vapoursynth would allocate a piece of memory to save the video frame raw data each time when get_frame() is called, but I am very puzzled that how videoNode.output() method wouldn't cause such a crash if the above statement is true. Is there anyone who can point out the reason, or give a solution to this problem?

There is a reference leak that's been fixed in svn that's causing it. I really should do a round of debugging and release r19 soon...

Have a new vapoursynth.pyd file without the leak (https://dl.dropboxusercontent.com/u/73468194/vapoursynth.pyd)

Tell me if it still leaks after this.

[IsoaSFlus]

sorry
when i executed "./waf build"
igot some errors:
Waf: Entering directory `/root/src/vapoursynth-read-only/build' Waf: Leaving directory `/root/src/vapoursynth-read-only/build' File /root/src/vapoursynth-read-only/src/core/asm/expr.asm has no ma pping in ['.c', '.o', '.obj', '.ts', '.C', '.qrc', '.c++', '.cpp', ' .ui', '.pyx', '.cc', '.pc.in', '.cxx'] (did you forget to load a waf tool?)
how can i solve this problem?
tks
(i use Ubuntu 13.04 32bit)

[jackoneill]

sorry
when i executed "./waf build"
igot some errors:
Waf: Entering directory `/root/src/vapoursynth-read-only/build' Waf: Leaving directory `/root/src/vapoursynth-read-only/build' File /root/src/vapoursynth-read-only/src/core/asm/expr.asm has no ma pping in ['.c', '.o', '.obj', '.ts', '.C', '.qrc', '.c++', '.cpp', ' .ui', '.pyx', '.cc', '.pc.in', '.cxx'] (did you forget to load a waf tool?)
how can i solve this problem?
tks
(i use Ubuntu 13.04 32bit)

Please paste all the commands you ran and their output, if possible.

[Adub]

@Myrsloik

Can I get your input on the bitdepth behavior of Lut2, as described above?

I'd like to get a patch for a working version of Lut2 submitted, but I'm just not sure about the desired handling behavior.

[Myrsloik]

@Myrsloik

Can I get your input on the bitdepth behavior of Lut2, as described above?

I'd like to get a patch for a working version of Lut2 submitted, but I'm just not sure about the desired handling behavior.

What I want:
A lut with a maximum of 20 bits input and selectable 8-16 bit output.
The two input clips can be between 8 and 12 bits in any combination that's less than 20 bits total.
I think that's all. This is probably a bit annoying to do without templates since you need to make several functions the different combinations of 8/16bit input.

[IsoaSFlus]

Please paste all the commands you ran and their output, if possible.
sorry,but what i posted is all info

[Adub]

@IsoaSFlus
No, it's not. You did not post your "./waf configure" step and any of the corresponding output, nor all output from the build step.

There should be a significant amount of terminal output from the configure and build steps. Be sure to use something like the forum formatting for easier reading.

[IsoaSFlus]

@IsoaSFlus
No, it's not. You did not post your "./waf configure" step and any of the corresponding output, nor all output from the build step.

There should be a significant amount of terminal output from the configure and build steps. Be sure to use something like the forum formatting for easier reading.
i see,tks a lot and
this is my config.log
hiso.comoj.com/something/config.log
and then,i built it

root@localhost:~/src/vapoursynth-read-only# ./waf build
Waf: Entering directory `/root/src/vapoursynth-read-only/build'
Waf: Leaving directory `/root/src/vapoursynth-read-only/build'
File /root/src/vapoursynth-read-only/src/core/asm/expr.asm has no ma
pping in ['.c', '.o', '.obj', '.ts', '.C', '.qrc', '.c++', '.cpp', '
.ui', '.pyx', '.cc', '.pc.in', '.cxx'] (did you forget to load a waf
tool?)
root@localhost:~/src/vapoursynth-read-only#

i think that ia the all,what should i do?

[sl1pkn07]

you uses root for build?

NEVER build with root. only use root to install (waf install)

[Are_]

Well, build is complaining it can't build asm files, you are on arm platform, and your log does not show any yasm/nasm check. I guess arm is not supported anyway.

[Myrsloik]

Only x86 is supported for now. Mostly because I don't have any arm stuff to test it on but porting should be a quick thing to do. Zor said that he'll try to port it to both arm and powerpc soon.

The real question is, why are you trying to run this on arm at all?

[IsoaSFlus]

@Myriloik
the answer is:
my senior high school doesn't allow us to bring pc to school...so,i only have a Android mobile.TAT.(i study at a boarding school )

[Adub]

A fixed version of Lut2 is available as a diff on the Google Code tracker.

[Myrsloik]

#!/usr/bin/env python3

import vapoursynth as vs
import sys

core = vs.Core()
std = core.std

def ivtc(clip, offsets, cycle=10, tff=True):
clip = std.SeparateFields(clip, tff)
clip = std.DoubleWeave(clip, tff)
return std.SelectEvery(clip, cycle, offsets)

def show_progress(c, t):
print('%i / %i' % (c, t), end='\r', file=sys.stderr, flush=True)

src = std.BlankClip(width=1920, height=1080, format=vs.YUV420P8, length=50000)

clips = []
for f in range(200):
clips.append(ivtc(src[f * 100: f * 100 + 30], [1, 4, 6, 9]))
clips.append(ivtc(src[f * 100 + 30: f * 100 + 60], [0, 2, 5, 7]))
clips.append(ivtc(src[f * 100 + 60: f * 100 + 100], [1, 3, 6, 8]))

last = std.Splice(clips)
#print(last, file=sys.stderr)
last.output(sys.stdout, False, 0, show_progress)


When I ran this script, the speed falls at about 6000th frame and the amount of the memory used continued increasing.
Is this bug or limitation ?

Definitely a bug. I'll see if I can find the leak...

[Adub]

I can confirm that r498 fixed the memleak issues with caching and Splice. Just ran it on my Ubuntu 13.04 machine.

[Myrsloik]

The next version is getting closer but I still want to add some easy way to access frame data from python. Currently you can use ctypes to get the raw data but I was thinking about maybe adding support for getting a frame as a numpy array or something similar. Any suggestions or alternative idea on how to do this?

[LoRd_MuldeR]

Any chance for an improved CLI front-end, which external programs could use to query the frame rate, size and count for user-provided VapourSynth scripts?

[Myrsloik]

Any chance for an improved CLI front-end, which external programs could use to query the frame rate, size and count?

That would be easy to add. How do you want the output?

Format: YUV422
Width: 100
Height: 200
Frames: 2344
FPS: 30000/1001

I could easily add something like that to my new cli output thing if that's what you need. And tell me if you need any other information to be in there.

[LoRd_MuldeR]

I would say that looks fine :)

Though I can imagine that an "interlaced vs. progressive" flag could be helpful for some applications too.

[mastrboy]

Maybe add bitdepth also?
If vapoursynth could signal for example 8/10bit to x264, that could perhaps spare us for the --input-depth parameter?

[Myrsloik]

Maybe add bitdepth also?
If vapoursynth could signal for example 8/10bit to x264, that could perhaps spare us for the --input-depth parameter?

I'll add it to the info, but it already can with y4m headers added.

[Myrsloik]

If someone wants to contribute a small patch to improve the lut/lut2 filters:
https://code.google.com/p/vapoursynth/issues/detail?id=52

[tin3tin]

If you haven't noticed it, I would just like to mention the great things happening with the Magic Lantern hack for Canon http://www.eoshd.com/content/10324/big-news-hands-on-with-continuous-raw-recording-on-canon-5d-mark-iii

The hack opens up for the possibility to shoot 14 bit raw with a camera within the consumer price range. Do you think an friendship between vapoursynth and ML RAW could evolve? (I mean import of RAW)

Here is a description of the current workflow and raw2dng code:
http://www.magiclantern.fm/forum/index.php?topic=5404.msg34757#msg34757

[Myrsloik]

Looks fairly easy to adapt, I suppose. However personally I'd rather work on more general filters than <insert very specific camera model here> raw format readers.

I'll help anyone who tries to write such a source filter.

[tin3tin]

Looks like they are aiming at the more common CinemaDNG as the format they what to convert to, but the Raw format hasn't found its final format yet I guess. What about the bitrate. What bitrates can Vapoursynth handle/convert to?

[Myrsloik]

Looks like they are aiming at the more common CinemaDNG as the format they what to convert to, but the Raw format hasn't found its final format yet I guess. What about the bitrate. What bitrates can Vapoursynth handle/convert to?

Uh? What?

VapourSynth only deals in uncompressed video, just like avisynth.

[qyot27]

How exactly is output treated now? With a generic SVN build of r508 on the Athlon64 / Ubuntu 13.04 64-bit system I have access to:

Script (going off of the example in doc/):
#!/usr/bin/python3
import vapoursynth as vs
import sys
core = vs.get_core()
core.std.LoadPlugin("/usr/local/lib/vslsmashsource.so")
v = core.lsmas.LibavSMASHSource("testing.mp4")
v.set_output()

Error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "vapoursynth.pyx", line 534, in vapoursynth.VideoNode.set_output (build/temp.linux-i686-3.3/pyrex/vapoursynth.c:9067)
vapoursynth.Error: 'Internal environment id not set. Was set_output() called from a filter callback?'

It does it with FFMS2 as well, so it doesn't seem to be plugin related (unless it's something deeper that plugins now need to account for?). The version of Cython is either 0.17.4 or 0.19.1; it throws the same error with either one of them.

[Myrsloik]

You need to use vspipe (which isn't added to the build system yet). Poke zor harder to catch up with my changes. I'll make a windows release tomorrow unless something unexpected happens and update the docs.

[sl1pkn07]

when make waf test (r508)

http://sl1pkn07.no-ip.com/paste/view/4e7ce5cd

greetings

[Myrsloik]

Have a test version of R19 (https://dl.dropboxusercontent.com/u/73468194/vapoursynth_r19_test.exe). I didn't really test it enough to call it a release yet but I think it works.

The biggest new thing is that it has a sane api for evaluating scripts and getting frames. Someone could make an editor similar to avspmod now... probably. Or actually embed it into another application.

Major changes in your scripts:
The python module now uses a singleton pattern:
vs.Core() => vs.get_core()
The core will be created with the options passed to the first get_core() call.
clip.output() and last have been replaced by clip.set_output() to make it uniform. Use vspipe.exe to pipe the contents of a script to stdout/whereever.

And test VSFS. I didn't really do that yet.

Changes:
r19:
calls to the getFrame() function inside vapoursynth can never deadlock now, the thread handling is also slightly improved
fixed a bug in the cache that would stop it from freeing all its frames when memory is needed, fixes an out of memory error reported by Chikuzen
output is no longer determined by setting the last variable in __main__, instead clip.set_output(int index = 0) is used to set which clip(s) are exported
fixed some memory leaks in the internal filters when they're freed (jackoneill)
now properly initializes blankclips to all 0 when no values are given
added proper const declarations now that cython supports it, cython 0.18 or later required to build the extension now
fixed get_write_ptr() in python to actually return a write pointer
fixed the lut and lut2 filters for > 8 bit formats (ADub)
various minor fixes to vivtc
removed clip.output() since it's a bad design decision, now a command-line program called vspipe can be used to pipe or write raw video output
renamed newMap() to createMap() to match the naming of all other functions
list_functions() renamed to get_plugins() for the core and get_functions() for namespaces, it also returns a dict instead of a string now
removed the unmentioned and unused link option for filter arguments as it is completely pointless and similar functionality may be implemented anyway
there's now a sane external api available for embedding in other applications, see vsscript.h and vsvfw for an example
now a singleton pattern is used for the core in python scripts, use vs.get_core() to get a reference to it
clip.get_frame() now throws an exception when out of bounds frames are requested
fixes a reference leak in the VideoProps python class
now ignores functions with . in their argument string for avisynth compatibility, this makes warpsharp mostly compatible

[LoRd_MuldeR]

Thanks for adding vspipe, that should make things much easier!

[sl1pkn07]

howto build vspipe in linux (wine)?

[Myrsloik]

What does wine have to do with linux?

[sl1pkn07]

then howto build vspipe? no need wine sources (to build on linux to linux)?

[jackoneill]

then howto build vspipe? no need wine sources (to build on linux to linux)?

No, you don't need wine for that.


clang++ -Wall -Wextra -Wno-unused-parameter -o libvsscript.so -shared -fPIC vsscript.cpp -I../../include -I../../build/temp.linux-x86_64-3.3/pyrex/ `pkg-config --cflags python3` `pkg-config --libs python3 vapoursynth`

and then

clang++ -Wall -Wextra -Wno-unused-parameter -o vspipe -fPIC vspipe.cpp -I../../include `pkg-config --cflags QtCore` -L../vsscript/ -lvsscript `pkg-config --libs QtCore`

Something like that. You can guess in what folders to run those.

[sl1pkn07]

thanks jackoneill

EDIT: none, don't see the last commit

greetings

[vdcrim]

Some issues of varying importance:


VfW: Reloading a script (e.g. F2 on VirtualDub) that loads any plugin will throw an exception 'XXX already loaded' (new in r19).


AddBorders: if the 'color' parameter is not given, a random color different each time is applied instead of [0, 0, 0] (AddBordersData's color is not initialized).


Calling VapourSynth from Python may crash

Not sure if this can be called a VapourSynth bug, but anyway: trying to use vsscript.dll (with ctypes) or evaluate the Avisynth script 'AviSource("script.vpy")' from Python may cause the process to crash at Py_Initialize depending on the environment. It seems that sys.path is populated at the Py_Initialize call from the caller's directories, not from Python 3.3.

Example of crash (Python 2):
import ctypes
ctypes.windll.vsscript.vseval_init()

I haven't managed to fully make clear this, but here are some notes:



If the Python application is wrapped with py2exe or similar all works as expected.
If not, the crash happens even if the environment variables PYTHONHOME and PYTHONPATH are not set.
If then PYTHONHOME is set pointing to the Python 3.3's directory, it works OK. I guess the same thing would happen if Py_SetProgramName is called before Py_Initialize.


If there's not a clean way of solving this then don't bother. I found the issue playing around without a defined purpose with the new vsscript API. If I or someone else adds VS support to AvsPmod it'll probably be by porting it to Python 3 (which is not possible yet) and using the included bindings instead.

[Myrsloik]

Interesting lump of bugs. I guess I'll rework the singleton thing a bit to avoid the issues.

The loading vsscript in python one will be an interesting one to try to figure out. Is this only when the python versions of vs and the loading script are different?

I suspect it's python that doesn't handle two different versions being loaded in the same process well. Exactly what did you wrap with py2exe?

[Chikuzen]

from VSScript.h

// Initialize the available scripting runtimes, returns non-zero on failure
VS_API(int) vseval_init(void);


I wasted my time about 30 minutes by this typo :(

[Myrsloik]

Huh? I though i wrote the code to work like the comment says. Guess I'll check it when i get home.

[vdcrim]

The loading vsscript in python one will be an interesting one to try to figure out. Is this only when the python versions of vs and the loading script are different?

Now that I try it with Python 3.3, it throws an 'OSError: exception: access violation reading 0x00000008', so sys.path is not the only problem.

If the script is called with Python 2.7 vsscript tries to load the Python resources from it:
Fatal Python error: Py_Initialize: unable to load the file system codec
File "C:\Program Files (x86)\Python27\lib\encodings\__init__.py", line 123
raise CodecRegistryError,\
^
SyntaxError: invalid syntax

Here's the incomplete bindings I wrote for playing around, in case it may be useful: https://gist.github.com/vdcrim/5891445
(https://gist.github.com/vdcrim/5891445)

I suspect it's python that doesn't handle two different versions being loaded in the same process well. Exactly what did you wrap with py2exe?
AvsPmod, either by adding the ctypes code to the application or as a macro script.

Also note that it also fails with 'AviSource("script.vpy")': if not using a py2exe build, it will crash.

Huh? I though i wrote the code to work like the comment says. Guess I'll check it when i get home.
I can confirm that Chikuzen is right.

[Myrsloik]

Have another test version (https://dl.dropboxusercontent.com/u/73468194/vapoursynth_r19_test2.exe).

It fixes everything reported apart from the odd python in python issue. I could really use some help with that one...

Changes: vseval_getCore(handle) <- now needs a script handle as argument
I also changed it so every script environment gets its own core which fixes a lot.
I simply changed the comment for vseval_init() to match current behavior
The color in addborders is now zero initialized.

Report anything else you find, I want this to be a working release once it's done.

[sl1pkn07]

stop build vspipe

http://sl1pkn07.no-ip.com/paste/view/0d18ac60

greetings

[Chikuzen]

>>> import vapoursynth as vs
>>> core = vs.get_core()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "vapoursynth.pyx", line 777, in vapoursynth.get_core (src\cython\vapoursy
nth.c:12897)
vapoursynth.Error: 'Internal environment id not set. Was get_core() called from
a filter callback?'
>>>

what's 'Internal environment id' ?

[Myrsloik]

>>> import vapoursynth as vs
>>> core = vs.get_core()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "vapoursynth.pyx", line 777, in vapoursynth.get_core (src\cython\vapoursy
nth.c:12897)
vapoursynth.Error: 'Internal environment id not set. Was get_core() called from
a filter callback?'
>>>

what's 'Internal environment id' ?

I forgot that scripts could also be called from outside of vsscript. I'll post a third test version a bit later today.

[Myrsloik]

Here's a third test (https://dl.dropboxusercontent.com/u/73468194/vapoursynth_r19_test3.exe).

Changes:
Possibly fixed the "python in python" crash, now it at least gets past vseval_init() on my computer (seemed to be caused by differences in GIL state)
Fixed the error that happend when not running scripts through vsscript
VSScript tells python to no install its signal handlers when loaded
Fixed the fps printed in vspipe's info

[mastrboy]

Myrsloik: What are the advantages of running a native compiled vapoursynth plugin versus loading a plugin with core.avisynth.LoadPlugin() ?

[vdcrim]

Updated list of head-scratchers:

The sys.path thing (for both vsscript and VfW). Other result of this (because is fixed by setting PYTHONHOME): if vsscript is called from Python 3.2 it crashes at vseval_evaluateScript.
Issues if vsscript is called from (and only from) Python 3.3: the OSError exception at vseval_init is fixed in test3, vseval_evaluateScript also passes (the gist linked a bit above is now Python 3 compatible) but the process gets stuck at vseval_finalize.

[Myrsloik]

Myrsloik: What are the advantages of running a native compiled vapoursynth plugin versus loading a plugin with core.avisynth.LoadPlugin() ?

If you don't need any features beyond what avisynth 2.5 offers and the plugin can't be easily changed to a better threading model (rare), nothing.

The key here is that x64 builds for windows are coming in a bit and then old avisynth stuff just won't work. (and don't think avisynth x64 plugins is the answer, since they're usually "delete key" ports and run slower than the 32-bit equivalents)

A properly written vs plugin also has the advantage of being compiled/used on any OS. I think of this as a new start where I and some other people slowly port/rewrite the most popular parts of avisynth's filters so they finally work in a modern environment everywhere...

[Chikuzen]

BTW, where did this feature go?
list_functions() renamed to get_plugins() for the core and get_functions() for namespaces,
it also returns a dict instead of a string now
new list_functions() is hard to use.

[Myrsloik]

BTW, where did this feature go?

new list_functions() is hard to use.

Hard to use in what way? What do you want changed? Should I restore the old list_functions() too for an easier to read format?

[Myrsloik]

Have a fourth test version (https://dl.dropboxusercontent.com/u/73468194/vapoursynth_r19_test4.exe).

This version restores list_functions(). get_functions()/get_plugins() is available to get dicts which are easier to deal with if programs want to enumerate everything.
It also fixes the "python in python" issue for python 3.3 and kinda other versions (insert huge asterisk here about vseval_finalize() hanging).

Keep testing. At this rate this will actually be a fairly stable release.

[Myrsloik]

Is the bug in Expr for > 8 bit formats reported by 06_taro (http://forum.doom9.org/showthread.php?p=1627283#post1627283) not to be fixed yet?

BTW, could the 'planes' argument of Lut/Lut2 be set to [0, 1, 2] by default(all planes will be processed), thus make it an optional argument?

I completely missed that expr was also mentioned. Will take a look at it tonight. Changing the default also makes sense i guess.

[zettai]

Has anyone managed to get access to a frame's image with vapoursynth via a numpy array or a pillow image or similar?

[Chikuzen]

Has anyone managed to get access to a frame's image with vapoursynth via a numpy array or a pillow image or similar?

I have previewed using PIL before since there was no VFW module at that time.


#vsshow.py
import vapoursynth as vs
from ctypes import *
from PIL import Image

def show(core, clip, frame_number, vflip=1):
format = clip.format.id
width = clip.width
height = clip.height
if format == vs.GRAY16:
clip = core.resize.Point(clip, width, height, vs.GRAY8)
if format != vs.GRAY8 and format != vs.RGB24:
clip = core.resize.Bicubic(clip, width, height, vs.COMPATBGR32)

format = clip.format.id
planes = range(clip.format.num_planes)
frame = clip.get_frame(frame_number)
data = [(frame.get_read_ptr(p), frame.get_stride(p)) for p in planes]
buff = [b'\0' * data[p][1] * height for p in planes]
for p in planes:
memmove(buff[p], data[p][0], data[p][1] * height)

if format == vs.COMPATBGR32:
mode, src = 'RGBA', 'BGRA'
else:
mode, src = 'L', 'L'
img = [Image.frombytes(mode, (width, height), buff[p], 'raw', src,
data[p][1], vflip) for p in planes]
if len(planes) != 1:
img = Image.merge('RGB', (img[2], img[0], img[1]))
else:
img = img[0]

img.show()

[Chikuzen]

@Myrsloyk
Would you add a function like avisynth's "Import" for VSScript?
It is annoy that the code for preparing a buffer, opening and reading a file and closing it must be written each time.

[Myrsloik]

@Myrsloyk
Would you add a function like avisynth's "Import" for VSScript?
It is annoy that the code for preparing a buffer, opening and reading a file and closing it must be written each time.

I guess I'll add a function to evaluate a script file as well.

Anyway, Zor finally updated the build system and ported VapourSynth to powerpc and arm. Have fun testing it on your ancient mac or something.

[vdcrim]

AviSynth's Import does a pair of extra things besides opening and evaluating the file:

Change the working directory to the script's parent directory before evaluating it (to allow use of relative paths).
Set some global variables: ScriptName, ScriptFile, ScriptDir.

Will VapourSynth do something similar? Maybe by passing an optional filename string to vseval_evaluateScript. That way future applications using evaluate instead of import won't have to care about mirroring this behaviour themselves.

[Groucho2004]

AviSynth's Import does a pair of extra things besides opening and evaluating the file:

Change the working directory to the script's parent directory before evaluating it (to allow use of relative paths).

No it does not. It sets the current directory to the directory in which the script resides.

[Chikuzen]

AviSynth's Import does a pair of extra things besides opening and evaluating the file:

Change the working directory to the script's parent directory before evaluating it (to allow use of relative paths).
Set some global variables: ScriptName, ScriptFile, ScriptDir.

Will VapourSynth do something similar? Maybe by passing an optional filename string to vseval_evaluateScript. That way future applications using evaluate instead of import won't have to care about mirroring this behaviour themselves.

My wish is contracting line 186 to 209 of this code (https://github.com/chikuzen/VapourSource/blob/master/src/VapourSource.cpp) to four lines like

if (vseval_new_function(&se, source)) {
env->ThrowError("VapourSource: failed to evaluate script.\n%s",
vseval_getError(se));
｝


of cource, without QtCore.

[vdcrim]

No it does not. It sets the current directory to the directory in which the script resides.
Isn't that what I said? :confused:

On second thought adding that functionality to vseval_evaluateScript may be a bad idea and doesn't solve the general problem: import might do things besides opening and evaluating the script (not necessarily limited to what AviSynth currently does), and someone using evaluate (because there's not file) would have to check the code and implement it themselves. Then the best would be adding to the API something like ImportFile and ImportBuffer instead of a single import.

[Groucho2004]

Isn't that what I said? :confused:
No. The script's parent directory is not the script's directory.

[Myrsloik]

I guess I'll add two new functions, vseval_importFile(&se, filename) and vseval_importBuffer(&se, scriptBuffer, imaginaryFilename) to do the things you mentioned.

I still want to keep the original function as well in case someone wants to have a different behavior.

[vdcrim]

No. The script's parent directory is not the script's directory.
Both are often used interchangeably. It could be just a common mistake though...

I still want to keep the original function as well in case someone wants to have a different behavior.
I agree.

[vdcrim]

It would be worth adding a comment somewhere in the header files saying that strings are always encoded/decoded with UTF-8.

[qyot27]

How do we set the correct python install so waf doesn't get confused? The addition of vsscript to the build system altered the detection method, and now it can only find 2.7.4 on my system. Setting it back to the old python detection allows it to pick up on 3.3.1, but it doesn't really solve it because the build process fails when it gets to vsscript.

I tried setting an alias so that 'python' is detected as python3, but waf still picks 2.7.4.

[Myrsloik]

PYTHON=python3 ./waf...
I think that will work. Note that i haven't managed to build it myself on Mac yet. I've also mentioned the python detection issues to zor so he should get around to fixing it soon.

[Adub]

I was able to build rev-513 on my Macbook Pro a little while back, but I haven't tested the latest updates quite yet (should be able to soon). Internet's been a little spoty recently so I haven't had a chance to sync yet.

That, and I also use a virtualenv for my dev purposes with just python3 loaded.

[Myrsloik]

I was able to build rev-513 on my Macbook Pro a little while back, but I haven't tested the latest updates quite yet (should be able to soon). Internet's been a little spoty recently so I haven't had a chance to sync yet.

That, and I also use a virtualenv for my dev purposes with just python3 loaded.

I did get it to build on mac now. Simply install qt, python3 and yasm using brew. Then install cython one way or another. Do PYTHON=python3. /waf... For all build commands.

Also, the project has moved to github now. Enjoy the absolutely zero difference in development efficiency...

[Adub]

I did see some issues when running ./waf test, even with PYTHON=python3. This is on Ubuntu 12.04, rev 530.

Here is my output:

blah@u1c6f650423c14ff77388:~/code/vapoursynth$ PYTHON=python3 ./waf test
Waf: Entering directory `/home/blah/code/vapoursynth/build'
Traceback (most recent call last):
File "/home/blah/code/vapoursynth/.waf-1.7.11-edc6ccb516c5e3f9b892efc9f53a610f/waflib/Scripting.py", line 97, in waf_entry_point
run_commands()
File "/home/blah/code/vapoursynth/.waf-1.7.11-edc6ccb516c5e3f9b892efc9f53a610f/waflib/Scripting.py", line 153, in run_commands
ctx=run_command(cmd_name)
File "/home/blah/code/vapoursynth/.waf-1.7.11-edc6ccb516c5e3f9b892efc9f53a610f/waflib/Scripting.py", line 146, in run_command
ctx.execute()
File "/home/blah/code/vapoursynth/.waf-1.7.11-edc6ccb516c5e3f9b892efc9f53a610f/waflib/Scripting.py", line 351, in execute
return execute_method(self)
File "/home/blah/code/vapoursynth/.waf-1.7.11-edc6ccb516c5e3f9b892efc9f53a610f/waflib/Build.py", line 106, in execute
self.execute_build()
File "/home/blah/code/vapoursynth/.waf-1.7.11-edc6ccb516c5e3f9b892efc9f53a610f/waflib/Build.py", line 109, in execute_build
self.recurse([self.run_dir])
File "/home/blah/code/vapoursynth/.waf-1.7.11-edc6ccb516c5e3f9b892efc9f53a610f/waflib/Context.py", line 128, in recurse
user_function(self)
File "/home/blah/code/vapoursynth/wscript", line 393, in test
if subprocess.Popen([ctx.env.PYTHON, name]).wait() != 0:
File "/usr/lib/python2.7/subprocess.py", line 679, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1249, in _execute_child
raise child_exception
AttributeError: 'list' object has no attribute 'rfind'

[Adub]

Ahhh oops!! Never mind, didn't see the new Github repo. The latest checkout from there builds and tests completely successfully.

For those of you wondering, the new source URL is here : https://github.com/vapoursynth/vapoursynth

[buchanan]

Hello Myrsloik

I'm using the 4th r19 test version, and i'm trying to run an x264 encode using the stdout method. My command line is:

"C:\Program Files (x86)\VapourSynth\core\vspipe.exe" "C:\test\test.vpy" - -y4m | "C:\test\x264.exe" --crf 20 --output "C:\test\test.mkv" --demuxer y4m -

The encoding runs fine, but at the end I get this message:

http://img11.hostingpics.net/pics/880324crash.png

(vspipe.exe stopped working, close the program or debug etc)

The output file seems to be correct though

That didn't happen with the 1st r19 test version, I didn't try with the 2nd and 3rd

[Myrsloik]

Hello Myrsloik

I'm using the 4th r19 test version, and i'm trying to run an x264 encode using the stdout method. My command line is:

"C:\Program Files (x86)\VapourSynth\core\vspipe.exe" "C:\test\test.vpy" - -y4m | "C:\test\x264.exe" --crf 20 --output "C:\test\test.mkv" --demuxer y4m -

The encoding runs fine, but at the end I get this message:

(vspipe.exe stopped working, close the program or debug etc)

The output file seems to be correct though

That didn't happen with the 1st r19 test version, I didn't try with the 2nd and 3rd

It's a known issue. I just haven't gotten around to fixing it yet. Pressing ctrl-c will also make it crash. It shouldn't do anything bad to the output since it crashes after everything is done.

[buchanan]

It's a known issue. I just haven't gotten around to fixing it yet. Pressing ctrl-c will also make it crash. It shouldn't do anything bad to the output since it crashes after everything is done.

Ok ! Thank you for your quick answer :)

[zerowalker]

Is vapoursynth supposed to succeed Avisynth?

From my understanding, it´s not, and just a "by product", but i wanted to make sure.

[Myrsloik]

Is vapoursynth supposed to succeed Avisynth?

From my understanding, it´s not, and just a "by product", but i wanted to make sure.

It's a by-product in the same way that humans are by-products of sex.

[ryrynz]

Myrsloik, I think you've just gone and made yourself a new signature.

VapourSynth (http://www.vapoursynth.com/) - a by-product of Avisynth in the same way that humans are by-products of sex.

[Adub]

Small bug I noticed, comment here: https://github.com/vapoursynth/vapoursynth/commit/244f23d244d55770a3c32258f25d1f523b1e2bdd#commitcomment-3620372 (https://github.com/vapoursynth/vapoursynth/commit/244f23d244d55770a3c32258f25d1f523b1e2bdd#commitcomment-3620372)

Essentially, commit 244f23d244d55770a3c32258f25d1f523b1e2bdd used the "FEATURE_AVISYNTH" syntax instead of "VS_FEATURE_AVISYNTH" syntax.

EDIT: Nevermind, it appears this was fixed.

[sl1pkn07]

waf test fail

http://sl1pkn07.no-ip.com/paste/view/0caf88ed

[Adub]

I'd wager you are out of date from the git repo. The tests were fixed 7 days ago: https://github.com/vapoursynth/vapoursynth/commit/83d9594fe86418f20b02bc6c110474ad76717848

[sl1pkn07]

ok, then my problem is other


howto make waf test before install the library?

[Adub]

You could use a virtualenv and a local prefix to install the libs to your home directory to act as a staging area.

So, setup a virtualenv for Python3, and then use the --prefix=/home/blah option during the ./waf configure phase to install all objects into your home directory.

[Myrsloik]

Grab what will hopefully be released as R19 here:
VapourSynth R19 rc1 (https://dl.dropboxusercontent.com/u/73468194/vapoursynth_r19_rc1.exe)

I think I've managed to fix all reported bugs so far as well. If there's anything still happening in this version then report it again because I've missed it. The vsscript api may have some extremely minor changes for R20 if I get more feedback.

Most big changes listed here:
r19:
lut/lut2 now processes all planes by default
ported to work on arm and powerpc (Zor)
fixed 9-16 bit format support in Expr
properly initialize color in addborders to 0
calls to the getFrame() function inside vapoursynth can never deadlock now, the thread handling is also slightly improved
fixed a bug in the cache that would stop it from freeing all its frames when memory is needed, fixes an out of memory error reported by Chikuzen
output is no longer determined by setting the last variable in __main__, instead clip.set_output(int index = 0) is used to set which clip(s) are exported
fixed some memory leaks in the internal filters when they're freed (jackoneill)
now properly initializes blankclips to all 0 when no values are given
added proper const declarations now that cython supports it, cython 0.18 or later required to build the extension now
fixed get_write_ptr() in python to actually return a write pointer
fixed the lut and lut2 filters for > 8 bit formats (ADub)
various minor fixes to vivtc
removed clip.output() since it's a bad design decision, now a command-line program called vspipe can be used to pipe or write raw video output
renamed newMap() to createMap() to match the naming of all other functions
added core.get_plugins() for the core and core.namepace.get_functions() for namespaces, it works similarly to list_functions() but instead returns the information in a dict
removed the unmentioned and unused link option for filter arguments as it is completely pointless and similar functionality may be implemented anyway
there's now a sane external api available for embedding in other applications, see vsscript.h and vsvfw for an example
now a singleton pattern is used for the core in python scripts, use vs.get_core() to get a reference to it
clip.get_frame() now throws an exception when out of bounds frames are requested
fixes a reference leak in the VideoProps python class
now ignores functions with . in their argument string for avisynth compatibility, this makes warpsharp mostly compatible

[buchanan]

Thanks !

Trying to load an avisynth plugin with core.avs.LoadPlugin gives me : "No attribute with the name avs exists. Did you mistype a plugin namespace?"

[Myrsloik]

Thanks !

Trying to load an avisynth plugin with core.avs.LoadPlugin gives me : "No attribute with the name avs exists. Did you mistype a plugin namespace?"

I managed to compile it without avisynth support due to some build system changes.
Here's RC2 (https://dl.dropboxusercontent.com/u/73468194/vapoursynth_r19_rc2.exe).

I also put a windows compile of AssVapour in the installer for those who want to try it out on windows easily.

[buchanan]

Another small problem, Chikuzen's port of FineSharp (http://forum.doom9.org/showthread.php?t=166524) now throws the error "Lut2: plane index out of range"

(Apologies if I should have posted it in Chikuzen's thread)

[Myrsloik]

Another small problem, Chikuzen's port of FineSharp (http://forum.doom9.org/showthread.php?t=166524) now throws the error "Lut2: plane index out of range"

(Apologies if I should have posted it in Chikuzen's thread)

I figured out why. I posted the necessary script change to Chikuzen's thread (http://forum.doom9.org/showpost.php?p=1638570&postcount=2). Ironically he submitted the VapourSynth patch that broke it.

[Reel.Deel]

Hey Myrsloik.

I wanted to try out AssVapour but I can't get it loaded.

>>> import vapoursynth as vs
>>> core = vs.get_core()
>>> core.std.LoadPlugin(path=r'C:\VapourSynth\AssVapour\assvapour.dll')

In the python command line I get this error:
---------------------------
python.exe - Entry Point Not Found
---------------------------
The procedure entry point strnlen could not be located in the dynamic link library msvcrt.dll.

In python shell I get this:
Traceback (most recent call last):
File "<pyshell#2>", line 1, in <module>
core.std.LoadPlugin(path=r'C:\VapourSynth\AssVapour\assvapour.dll')
File "vapoursynth.pyx", line 978, in vapoursynth.Function.__call__ (src\cython\vapoursynth.c:15485)
vapoursynth.Error: 'Failed to load C:\\VapourSynth\\AssVapour\\assvapour.dll'

Hope I'm not doing something silly. :o

[Chikuzen]

Another small problem, Chikuzen's port of FineSharp (http://forum.doom9.org/showthread.php?t=166524) now throws the error "Lut2: plane index out of range"

(Apologies if I should have posted it in Chikuzen's thread)

for r19 or later
http://pastebin.com/1a0yXNbJ
(Although a reason is not known, I cannot update Gist now.)

import vapoursynth as vs
import finesharp
core = vs.get_core()
core.avs.LoadPlugin('/path/to/RemoveGrain.dll')
core.avs.LoadPlugin('/path/to/Repair.dll')
clip = something
fs = finesharp.FineSharp() #You do not need to set core any longer.
clip = fs.sharpen(clip, other options...)
clip.set_output()

[Myrsloik]

Hey Myrsloik.

I wanted to try out AssVapour but I can't get it loaded.

...

Hope I'm not doing something silly. :o

It was very miscompiled. lachs0r is making a second attempt at it now...

[vdcrim]

Grab what will hopefully be released as R19 here: [...]

I think I've managed to fix all reported bugs so far as well. If there's anything still happening in this version then report it again because I've missed it. The vsscript api may have some extremely minor changes for R20 if I get more feedback.

So, are these two things (http://forum.doom9.org/showpost.php?p=1635551&postcount=847) dismissed?

Also, vsscript_evaluateFile assumes that the input file is UTF-8 encoded. I know that this is because vsscript_evaluateScript expects an UTF-8 encoded string, but in order to avoid future "why my script doesn't work" complaints maybe it would be better to take the ugly way and change it so that the file is decoded first with the right codec (something like check for BOM -> if not found assume UTF-8 and decode it -> if decoding fails decode with the system's encoding) and encoded back to UTF-8 before passing it to vsscript_evaluateScript. Or just put somewhere EVERYTHING IS UTF-8.

[Myrsloik]

So, are these two things (http://forum.doom9.org/showpost.php?p=1635551&postcount=847) dismissed?

Also, vsscript_evaluateFile assumes that the input file is UTF-8 encoded. I know that this is because vsscript_evaluateScript expects an UTF-8 encoded string, but in order to avoid future "why my script doesn't work" complaints maybe it would be better to take the ugly way and change it so that the file is decoded first with the right codec (something like check for BOM -> if not found assume UTF-8 and decode it -> if decoding fails decode with the system's encoding) and encoded back to UTF-8 before passing it to vsscript_evaluateScript. Or just put somewhere EVERYTHING IS UTF-8.

Those two things aren't dismissed, I'm just saving it for a later version. I want to release this greatly improved version this week.

And yes. EVERYTHING IS UTF-8. Just like the encoding assumed by python 3 for all scripts everywhere. So now it works like python, that's good enough for me...

[Myrsloik]

Have a go at RC3 (https://dl.dropboxusercontent.com/u/73468194/vapoursynth_r19_rc3.exe). Should fix everything reported so far.

AssVapour is still broken though...

[aegisofrime]

Sorry if this is a dumb question, but I have been getting this error since Test 4 I believe, and I'm still getting it in RC3. I have looked over the changelog for anything that might have to do with this to no avail.

http://i.imgur.com/eUwKGoL.png

[Myrsloik]

Sorry if this is a dumb question, but I have been getting this error since Test 4 I believe, and I'm still getting it in RC3. I have looked over the changelog for anything that might have to do with this to no avail.

http://i.imgur.com/eUwKGoL.png

You're using an old copy of vspipe.exe. Delete all copies and reinstall.

[aegisofrime]

You're using an old copy of vspipe.exe. Delete all copies and reinstall.

So that's it! Thanks, it works now! :)

[Myrsloik]

R19 has been released. See the usual blog post (http://www.vapoursynth.com/2013/07/r19-finally-some-progress/) for notes and stuff. Changelog in the first post in the thread.

[sneaker_ger]

No AssVapour?

[Myrsloik]

No AssVapour?

No, lachs0r hasn't made a working compile yet. I was hoping to have it ready too but it needs lots of libraries that kinda hate windows. And a very recent mingw. It should be fairly easy to compile for linux and osx users though...

I'm going to poke him every day until there's a working compile.

[Mystery Keeper]

What about x64 versions of both VapourSynth and its plug-ins? Very complex scripts call for larger memory usage, and AviSynth crashes whenever used memory approaches 4GB even in x64 version.

[Myrsloik]

What about x64 versions of both VapourSynth and its plug-ins? Very complex scripts call for larger memory usage, and AviSynth crashes whenever used memory approaches 4GB even in x64 version.

Next version will most likely have x64 builds for windows. I actually do write my general plan for the next version on the blog...

If you want to try x64 stuff right now run vs on linux or mac.

[LoRd_MuldeR]

Today I have added VaporSynth support to my x264 GUI and I have to say VSPipe works exactly as expected :)

Anyway, I have two more small feature requests:
1. Add an option like "-version" to VSPipe so the installed VaporSynth version can be checked easily.
2. Automatically convert RGB formats to YUV when "-y4m" switch is used rather than aborting.

:thanks:


BTW: What is the recommend way to detect VapourSynth on Windows? I currently use the "Uninstall" registry entry to find the "InstallLocation". Is that reliable?

[Chikuzen]

BTW: What is the recommend way to detect VapourSynth on Windows? I currently use the "Uninstall" registry entry to find the "InstallLocation". Is that reliable?


HMODULE h = LoadLibrary("vsscript");
if (h) {
installed
} else {
not installed
}

probably, this is the easiest way.
There is no need to find where vapoursynth.dll was installed since you can access VS via vsscript.

[LoRd_MuldeR]

Didn't realize "vsscript.dll" gets installed into the system folder. Anyway, I also need to find "vspipe.exe" for my application.

Or do you suggest "vspipe.exe" should be redistributed with my application? Will future versions of "vsscript.dll" be backward compatible to current "vspipe.exe" version?

[Myrsloik]

Today I have added VaporSynth support to my x264 GUI and I have to say VSPipe works exactly as expected :)

Anyway, I have two more small feature requests:
1. Add an option like "-version" to VSPipe so the installed VaporSynth version can be checked easily.
2. Automatically convert RGB formats to YUV when "-y4m" switch is used rather than aborting.

:thanks:


BTW: What is the recommend way to detect VapourSynth on Windows? I currently use the "Uninstall" registry entry to find the "InstallLocation". Is that reliable?

1. Will be added.
2. Will never happen, converting behind the user's back will cause more problems than it solves. Why not add a conversion line to the end of the script when needed? It also sounds like you want to convert ONLY to conveniently pass clip metadata...
3. Currently there's no good way. I'll create proper registry keys in the R20 installer so HKLM\Software\VapourSynth will have keys containing paths and versions. Chikuzen's way will work if all you want to know is if it's R19 or later.

The path in the uninstall entry is of course also valid to use.

[LoRd_MuldeR]

2. Will never happen, converting behind the user's back will cause more problems than it solves. Why not add a conversion line to the end of the script when needed? It also sounds like you want to convert ONLY to conveniently pass clip metadata...

Wouldn't say it's "behind the user's back", when the caller requested "-y4m " explicitly. But maybe an additional "-force-y4m" switch could be added to avoid any possible confusion?

And indeed, I need to output YUV4MPEG, so that x264 gets the required meta information. Surely the required conversion line can be added to the script manually. But the problem here is that the script is provided by the user. And the GUI needs to ensure it will encode properly. It's not trivial at all to add the required line into an arbitrary user-provided script in an automated way. Currently it will simply error out. And then the user will have to understand and fix the issue himself. That's where 2/3 of all users say "Damn, the GUI isn't working with my script, let's try a different one!".

Also, I think in 99% of all use cases where the output of the script is sent to x264, we are going to need YUV anyway. I know x264 can do RGB encoding too, but that's a very rare use case. So, in 99.9% of all cases, either we sent YUV data to x264 or x264 will convert the RGB data to YUV by itself. This means forcing the RGB to YUV conversion on the VapourSynth side wouldn't hurt...


1. Will be added.
3. Currently there's no good way. I'll create proper registry keys in the R20 installer so HKLM\Software\VapourSynth will have keys containing paths and versions. Chikuzen's way will work if all you want to know is if it's R19 or later

:thanks:

[Reel.Deel]

@Myrsloik

I'm trying to write (more like copy-paste) documentation for AVISource and have some questions.

In Avisynth you can do this AVISource("video1.avi", "video2.avi"), is this possible in Vapoursynth? I tried but I get an error. Just want to make sure so I don't included.

Is the order of decompression still true or has it changed because of the additional color spaces that it now supports?
If pixel_type is omitted or set to "FULL", AviSynth will use the first format supported by the decompressor (in the following order: YV24, YV16, YV12, YV411, YUY2, RGB32, RGB24 and Y8).
If set to "AUTO", AviSynth will use the old ordering: YV12, YUY2, RGB32, RGB24 and Y8.

One last thing, I tried opening a vpy script through AVISource in VS but I get this message:
Python exception: 'Internal environment id not set. Was set_output() called from a filter callback?'


Is that correct?

[Myrsloik]

@Myrsloik

I'm trying to write (more like copy-paste) documentation for AVISource and have some questions.

In Avisynth you can do this AVISource("video1.avi", "video2.avi"), is this possible in Vapoursynth? I tried but I get an error. Just want to make sure so I don't included.

Is the order of decompression still true or has it changed because of the additional color spaces that it now supports?


One last thing, I tried opening a vpy script through AVISource in VS but I get this message:


Is that correct?

You can't pass multiple filenames. This is because internally in avisynth it's implemented as avisource(file1)++avisource(file2)++.. so there's no reason to copy avisynth here and reimplement splice badly.

Pixel_type works the same as avisynth describes but with a few additional formats.

unset tries (in this order):
YV24
YV16
YV12
YV411
YUY2
RGB32
RGB24
RGB48
Y8
P010
P016
P210
P216
v210

I'm going to remove AUTO and FULL (since FULL is implicit when unset) and AUTO is an avisynth leftover. No idea why anyone'd want the old order.

Interesting bug find. I think I know why it gives the environment id not set error and I'll probably fix it in the next version.

[lansing]

can you add the vspipe.exe to the environment variable path upon install, so we don't have to type the whole path every time?

[mp3dom]

Probably I'm doing something wrong, but I'm having problems adding black borders to a clip via AddBorders. The borders are always green. I've tried to add the command to a P210 clip but also on a YUV422P8 clip. The size is 1920x804 so I need to add 138px at top/bottom to pad the fullhd resolution.

[Myrsloik]

Probably I'm doing something wrong, but I'm having problems adding black borders to a clip via AddBorders. The borders are always green. I've tried to add the command to a P210 clip but also on a YUV422P8 clip. The size is 1920x804 so I need to add 138px at top/bottom to pad the fullhd resolution.

For 8bit:
AddBorders(..., color=[0,128,128])
For 10bit:
AddBorders(..., color=[0,512,512])

I'll make it default to black borders always in the next version.

[mp3dom]

Yup! I thought that default was black. Sorry to have not discovered it by myself reading the docs. Thanks for the support!

[IsoaSFlus]

hello Myrsloik,how can vapoursynth preview the video frame by frame in linux?

[Myrsloik]

hello Myrsloik,how can vapoursynth preview the video frame by frame in linux?

May i suggest wine and virtualdub?

[jackoneill]

hello Myrsloik,how can vapoursynth preview the video frame by frame in linux?

Maybe you could try this: https://github.com/dubhater/vapoursynth-viewer
(Tested with a grand total of 1 scripts.)

[IsoaSFlus]

Maybe you could try this: https://github.com/dubhater/vapoursynth-viewer
(Tested with a grand total of 1 scripts.)

i have gited it and compiled it,...and got a lot of error.i have posted the issue.

[IsoaSFlus]

umm...where can i get the introduction of assvapour?

[jackoneill]

umm...where can i get the introduction of assvapour?

I will assume you mean a guide...


clip = core.ffms2.Source("bigbuckbunny.mkv")

# Both AssRender() and Subtitle() return a list of two clips:
# the rendered subtitles in RGB24 and a Y8 mask you'll
# need to use to blend the subtitles with the video

clips = core.assvapour.AssRender(clip, "moo.ass")
# or
#clips = core.assvapour.Subtitle(clip, "Text to be burnt into the video.")

# convert the subtitles into the video's format (here we pretend it's YUV420P8)
clips[0] = core.resize.Bicubic(clips[0], format=vs.YUV420P8)

# and blend
ret = core.std.MaskedMerge([clip, clips[0]], clips[1])

[IsoaSFlus]

I will assume you mean a guide...


clip = core.ffms2.Source("bigbuckbunny.mkv")

# Both AssRender() and Subtitle() return a list of two clips:
# the rendered subtitles in RGB24 and a Y8 mask you'll
# need to use to blend the subtitles with the video

clips = core.assvapour.AssRender(clip, "moo.ass")
# or
#clips = core.assvapour.Subtitle(clip, "Text to be burnt into the video.")

# convert the subtitles into the video's format (here we pretend it's YUV420P8)
clips[0] = core.resize.Bicubic(clips[0], format=vs.YUV420P8)

# and blend
ret = core.std.MaskedMerge([clip, clips[0]], clips[1])


thank you very much~i rendered ass by assvapour successfully.

[Rodger]

Could some be so nice to translate the following easy script for me please?

Obviously I need to learn a loooot more about it as all my attempts are quitted with error code 1 :o (using Simple x264 launcher)


Loadplugin("C:\Program Files (x86)\AviSynth 2.5\plugins\DGDecodeNV.dll")

DGSource("The Score.dgi", resize_w=720, resize_h=576, deinterlace=1)

[Myrsloik]

Could some be so nice to translate the following easy script for me please?

Obviously I need to learn a loooot more about it as all my attempts are quitted with error code 1 :o (using Simple x264 launcher)

Something like this I guess:

import vapoursynth as vs
core = vs.get_core()
core.avs.LoadPlugin(r"C:\Program Files (x86)\AviSynth 2.5\plugins\DGDecodeNV.dll")
core.avs.DGSource(r"The Score.dgi", resize_w=720, resize_h=576, deinterlace=1).set_output()

I don't know if dgdecodenv works at all though. Script not actually tested.

[LoRd_MuldeR]

import vapoursynth as vs
core = vs.get_core()
core.avs.LoadPlugin(r"C:\Program Files (x86)\AviSynth 2.5\plugins\DGDecodeNV.dll")
core.avs.DGSource(r"The Score.dgi", resize_w=720, resize_h=576, deinterlace=1).set_output()

I don't know if dgdecodenv works at all though. Script not actually tested.

I can confirm it does work :)

[Rodger]

Something like this I guess:

import vapoursynth as vs
core = vs.get_core()
core.avs.LoadPlugin(r"C:\Program Files (x86)\AviSynth 2.5\plugins\DGDecodeNV.dll")
core.avs.DGSource(r"The Score.dgi", resize_w=720, resize_h=576, deinterlace=1).set_output()

I don't know if dgdecodenv works at all though. Script not actually tested.

THANK YOU!

I was close...not that close, but close :p
need to learn more about that "r" in the path setting

/EDIT: compared varoursynth with avisynth64 on my machine.
Very close....9560 frames with VS and 9575 with AS (5 minute timerun)
When there are 64bit builds available tell me I´ll repeat my test.

[Myrsloik]

I found that TDeint runs about 10x~20x slower in VS than AviSynth. Is it because TDeint was not in the prefetch list of avisynth_compat.cpp? Or it has nothing to do with that?

Give me a full example script and I'll try it here.

[Myrsloik]

import vapoursynth as vs
core = vs.get_core()
core.avs.LoadPlugin(r'C:\Program Files (x86)\AviSynth 2.5\plugins\DGDecodeNV.dll')
core.avs.LoadPlugin(r'C:\Program Files (x86)\AviSynth 2.5\plugins\TDeint.dll')
clip = core.avs.DGSource(r'D:\test.dgi') # a 1920x1080 video
clip = core.avs.TDeint(clip, mode=1, order=1)
clip.set_output()


I don't have an nvidia card. Does it also happen with dgdecode or any other source filter?

[Reel.Deel]

I encountered something odd with AVISource.

I have this video named "ashCIMG0636.avi".
If I try to open it in VDub with the following script I get an error.

import vapoursynth as vs
core = vs.get_core()
core.std.LoadPlugin(path=r'C:\Vapoursynth\AVISource\avisource.dll')

# Source
last = core.avisource.AVISource("C:\VapourSynth\AVISource\ashCIMG0636.avi", pixel_type="YUY2", fourcc="MJPG")

# Output
last.set_output()


Avisynth open failure:
Python exception: "AVISource autodetect: couldn't open file 'C:\\VapourSynth\\AVISource\x07shCIMG0636.avi'\nError code: 123"


If I change the name to "ASHCIMG0636.avi" it works fine.

[Myrsloik]

I encountered something odd with AVISource.

I have this video named "ashCIMG0636.avi".
If I try to open it in VDub with the following script I get an error.

import vapoursynth as vs
core = vs.get_core()
core.std.LoadPlugin(path=r'C:\Vapoursynth\AVISource\avisource.dll')

# Source
last = core.avisource.AVISource("C:\VapourSynth\AVISource\ashCIMG0636.avi", pixel_type="YUY2", fourcc="MJPG")

# Output
last.set_output()



If I change the name to "ASHCIMG0636.avi" it works fine.

You forgot to escape the \ properly. Put r before the string.

[Reel.Deel]

Thanks, it works. :)
Maybe the error message could be a little more informational.

Edit:
Lesson learned.

[Myrsloik]

Thanks, it works. :)
Maybe the error message could be a little more informational.

It can't. This is a python thing. Same thing as accidentally doing "\new\trees" in C... So it's simply impossible to detect. I warn about it everywhere in the documentation and you should always use / for paths anyway to please the slash gods. Especially relative ones because then scripts will work in more than windows...

[Myrsloik]

When there is an error/exception occured, the message vspipe shows does not indicate the line number that caused the error/exception. I think that's quite inconvenient to debug.;)

I'll improve the error message everywhere. It's not only vspipe.

[Myrsloik]

Yes. It's not related to source filter. Tried with lsmashsource(r666).

Here's a new dll for you to try. I added TDeint to the prefetch stuff.

vapoursynth.dll (https://dl.dropboxusercontent.com/u/73468194/vapoursynth.dll)

[jpsdr]

Sorry to ask stupid version, but i don't find clear answers.
- Is vapoursynth 64bit ? Does the installer install a 64bit version ?
In the first posts it seems to be the case, but on the site, no information, and it's asked to download a 32bit version of python.
I'm confused...
- Can a vapoursynth script be opened with vdub64 ? Can a vapoursynth script be use as input with a 64bit version of x264 ?

[sneaker_ger]

VapourSynth can be compiled for 64 bit, but Myrsloik does not offer a pre-compiled build for that at the moment. According to the website he does plan to offer Windows x64 builds starting with the next version, though. You can use vspipe.exe to pipe 32 bit scripts into x264 64 bit (similar to AviSynth). I don't think you can use 32 bit VS with vdub64.

[Myrsloik]

VapourSynth can be compiled for 64 bit, but Myrsloik does not offer a pre-compiled build for that at the moment. According to the website he does plan to offer Windows x64 builds starting with the next version, though. You can use vspipe.exe to pipe 32 bit scripts into x264 64 bit (similar to AviSynth). I don't think you can use 32 bit VS with vdub64.

May as well explain my reasons. The first one is that unless VS is feature complete enough no one will use it anyway. It won't be until the next version it will have a true equivalent to avisynth's scriptclip (and thus be able to express every avisynth script in existence). So without the basic features in place no one will care if it's 64 bits or not.

The next issue is that once you go x64 you lose all avisynth plugin compatibility as well. Without a base of portable plugins there'd once again be nothing to do with VS. That's the other reason I haven't posted any 64 bit builds yet.

There will be a public 64 bit build soon. (read: next month or so)

Linux and mac os has had 64 bit support from day one since there's no pile of legacy stuff to take into consideration. (don't mention avxsynth or I'll kill you all)

And no, of course you can't use a 32 bit dll inside a 64 bit application. So vdub64 will fail. The real question is why you're using vdub at all.

[jpsdr]

My main purpose is to see if there was an alternative for avisynth64, for input with x264 (64bit). I don't know what is vspipe, and i'm looking for a complete 64bit chain. But, for this to works, x264 (whatever version) should at least be able to work with vapoursynth, and from what i've found, apparently it doesn't, but, maybe i've missed something.

[LoRd_MuldeR]

Using 64-Bit x264 with 32-Bit Avisynth (via Avs2YUV) or 32-Bit VapourSynth (via vspipe) is implemented in Simple x264 Launcher, for example.

[sneaker_ger]

You can find the vspipe.exe in the VapourSynth directory ("C:\Program Files (x86)\VapourSynth\core\vspipe.exe"), it works like this:
vspipe.exe script.vpy - -y4m | x264_64.exe - --demuxer y4m -o output.h264

[jpsdr]

Just a little question.
I'm often working with YV12 video, and they stay YV12 in all my chain process, and all my final files are lossless YV12 avi files. For now, using direct avs input files, i'm sure that video is decoded in YV12 format and directly feed this way to x264 without no conversions or data modifications. Can the solutions of vspipe or avs2yuv guarantee me the exact same behavior ?

[sneaker_ger]

The piping solutions don't alter the output - the data is exactly the same as if you were directly opening the scripts with x264.

[Mystery Keeper]

May as well explain my reasons. The first one is that unless VS is feature complete enough no one will use it anyway. It won't be until the next version it will have a true equivalent to avisynth's scriptclip (and thus be able to express every avisynth script in existence). So without the basic features in place no one will care if it's 64 bits or not.

The next issue is that once you go x64 you lose all avisynth plugin compatibility as well. Without a base of portable plugins there'd once again be nothing to do with VS. That's the other reason I haven't posted any 64 bit builds yet.

There will be a public 64 bit build soon. (read: next month or so)

Linux and mac os has had 64 bit support from day one since there's no pile of legacy stuff to take into consideration. (don't mention avxsynth or I'll kill you all)

And no, of course you can't use a 32 bit dll inside a 64 bit application. So vdub64 will fail. The real question is why you're using vdub at all.

Actually, if there was ready x64 version, it could encourage plugin makers to make proper x64 plugins. They really need that encouragement. AviSynth (even x64 build) can not be used to its full potential because with many threads it hits the memory ceiling and crashes.

As for VirtualDub - the reason *I* use it is intermediate encode. My scripts are slow. So I make only one pass and encode in Lagarith with VirtualDub and only then re-encode with x264.

[TurboPascal7]

Actually, if there was ready x64 version, it could encourage plugin makers to make proper x64 plugins. They really need that encouragement. AviSynth (even x64 build) can not be used to its full potential because with many threads it hits the memory ceiling and crashes.
Avisynth x64 failed because there was no one crazy enough to rewrite the insane amount of x64-incompatible asm avisynth filters run on. It's not like people like this will suddenly appear when vsynth gets x64 support on windows. People who want to write x64-compatible code have been doing it for a long time.

I do agree that x64 build would be nice, but I do not think it will be useful any time soon and it definitely shouldn't be the top priority.

[zettai]

Is vspipe the only way to pipe data to programs like ffmpeg or is it possible to do it within Python using subprocess, for example?

I recall in earlier versions there was .output(sys.stdout, y4m=True) but this is no longer the case, is that correct or am I missing something?

[Myrsloik]

Is vspipe the only way to pipe data to programs like ffmpeg or is it possible to do it within Python using subprocess, for example?

I recall in earlier versions there was .output(sys.stdout, y4m=True) but this is no longer the case, is that correct or am I missing something?

It's no longer possible because it was a kinda bad idea to have different output methods. Also the GIL in python would've made complicated scripts a lot slower down the line.

You can either get the raw frame data:
frame = clip.get_frame(n)
frame.get_read_ptr(plane_number)
and do it yourself with ctypes

Or just be clever with your pipes... exactly what are you trying to do?

[zettai]

It's no longer possible because it was a kinda bad idea to have different output methods. Also the GIL in python would've made complicated scripts a lot slower down the line.

You can either get the raw frame data:
frame = clip.get_frame(n)
frame.get_read_ptr(plane_number)
and do it yourself with ctypes

Or just be clever with your pipes... exactly what are you trying to do?

Thanks for the info. I'm trying to see how easy it will be to build applications using vs and python, applications that I can freeze with cx_freeze or similar then distribute but if the python script is frozen, I'm not really sure how I could pipe it to ffmpeg.

I'm not sure I'm explaining this well so lets imagine a simple program. Say I have a python script which opens a tkinter filechooser to locate a file then, based on the file's mimetype, I load the appropriate import plugin, import the file as a source into vapoursynth, resize it in and then pipe it to ffmpeg to make an mp4 or something. With the old output function, I could do that by just using subprocess.PIPE (possibly) but now the python script itself is piped using vspipe I don't really know how I'd write this simple program without programmatically writing a python file which I then pipe with vspipe which feels wrong and doesn't solve my wish to make a standalone application. Does that make sense?

Is the only solution to write the code myself in python to pipe via subprocess using ctypes or is there something I'm missing?

[ganymede]

exactly what are you trying to do?It would also enable interactive scripting, which IMHO would be especially useful with a powerful interactive environment like ipython (http://ipython.org/). The python console could act for vapoursynth somewhat like AvsPmod for avisynth. vapoursynth scripting would be easier to learn, debug, etc. Interactivity is one of the benefits of using python, after all.

[Myrsloik]

It would also enable interactive scripting, which IMHO would be especially useful with a powerful interactive environment like ipython (http://ipython.org/). The python console could act for vapoursynth somewhat like AvsPmod for avisynth. vapoursynth scripting would be easier to learn, debug, etc. Interactivity is one of the benefits of using python, after all.

Blah
Blah
Blah
subprocess.call('vspipe.exe', __file__ + ' -', stdout=...)


As for the interactive part the whole script would still be syntax checked. I usually run my scripts in python to get better error information even if I don't get any output. (better error information will be in the next version)

[zettai]

Blah
Blah
Blah
subprocess.call('vspipe.exe', __file__ + ' -', stdout=...)



Sure this works fine but it doesn't help when I want to freeze the program and distribute it as an application. Once it's frozen there is no __file__ only sys.executable and I'm pretty sure vspipe has no way of doing anything with that.


Will I have to roll my own output pipe method?

[ganymede]

subprocess.call('vspipe.exe', __file__ + ' -', stdout=...)Unfortunately, __file__ is undefined in an interactive console, it exists only... in files.
If I enter some vapoursynth code into a python/ipython console, without saving it in a file, is there a way to interactively pipe the clip's (VideoNode) data to an external process ?
IMHO it would be useful to interactively display the clip (individual frames could even be displayed directly in the ipython qtconsole or notebook), pipe to an encoder, etc.

[Yellow_]

Hi, I'm trying to build Vapoursynth on Linux Ubuntu 13.04 64bit, have python 2.7 & 3.1, multitude of python dev libraries and what not, :-) but getting this error running the .bootstrap.py file:

Traceback (most recent call last):
File "./bootstrap.py", line 3, in <module>
import os, stat, urllib.request
ImportError: No module named request


Any help would be appreciated, I've searched doom9 and Google. I have the following python related stuff, possibly related to 'request' installed amongst a tonne of other python related stuff:

python-urllib3 1.5-0ubuntu1
python3-urllib3 1.5-0ubuntu1
python-multipartposthandler 0.1.0-2
python-restkit 4.1.3-2
python-requests 1.1.0-1
python3-requests 1.1.0-1

Typing python in a console tells me I'm using 2.7.4, 5th July 2013

:-)

[Myrsloik]

Hi, I'm trying to build Vapoursynth on Linux Ubuntu 13.04 64bit, have python 2.7 & 3.1, multitude of python dev libraries and what not, :-) but getting this error running the .bootstrap.py file:



Any help would be appreciated, I've searched doom9 and Google. I have the following python related stuff, possibly related to 'request' installed amongst a tonne of other python related stuff:

python-urllib3 1.5-0ubuntu1
python3-urllib3 1.5-0ubuntu1
python-multipartposthandler 0.1.0-2
python-restkit 4.1.3-2
python-requests 1.1.0-1
python3-requests 1.1.0-1

Typing python in a console tells me I'm using 2.7.4, 5th July 2013

:-)

You always have to explicitly use python 3:
python3 bootstrap.py
PYTHON=python3 ./waf configure
PYTHON=python3 ./waf build

or something along those lines... I don't build vs on linux every day

[Yellow_]

Myrsloik, that solved it, thank you. :-)

**EDIT**

After compiling and installing just tripping up at:

To install the Cython wrapper:

$ sudo PYTHON=python3 ./setup.py install

Gives me an error:

running install
running build
running build_ext
cythoning src/cython/vapoursynth.pyx to build/temp.linux-x86_64-2.7/pyrex/vapoursynth.c

Error compiling Cython file:
------------------------------------------------------------
...
arInitial = 0
arFrameReady = 1
arAllFramesReady = 2
arError = -1

ctypedef void (__stdcall *VSFrameDoneCallback)(void *userData, const VSFrameRef *f, int n, VSNodeRef *node, const char *errorMsg)
^
------------------------------------------------------------

src/cython/vapoursynth.pxd:147:84: Expected ')', found '*'

Error compiling Cython file:
------------------------------------------------------------
...
instance.core = core
instance.func = None
instance.ref = ref
return instance

cdef object mapToDict(const VSMap *map, bint flatten, bint add_cache, Core core, const VSAPI *funcs):
^
------------------------------------------------------------

src/cython/vapoursynth.pyx:157:34: Expected ')', found '*'
building 'vapoursynth' extension
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I. -Isrc/cython -I/usr/include/python2.7 -c build/temp.linux-x86_64-2.7/pyrex/vapoursynth.c -o build/temp.linux-x86_64-2.7/build/temp.linux-x86_64-2.7/pyrex/vapoursynth.o
build/temp.linux-x86_64-2.7/pyrex/vapoursynth.c:1:2: error: #error Do not use this file, it is the result of a failed Cython compilation.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1


**EDIT**

argh, maybe I've been here before, my cython version may be too old, 0.17 according to this post:

http://forum.doom9.org/showthread.php?p=1605356#post1605356

So just updating via pip and rebuilding. :-)

hehe, that was it, compiled and installed this time with cython 0.19.1 :-)

**EDIT**

Nope, still not working, after doing the sudo PYTHON=python3 .setup.py cython stuff and running PYTHON=python3 ./waf test in the console is giving me:

Traceback (most recent call last):
File "test/test.py", line 2, in <module>
import vapoursynth as vs
ImportError: No module named 'vapoursynth'


But maybe it's due to this, I'm no coder so may not be related, 2.7 is mentioned a lot below, so for so much text, really should use PasteAll or something:

running install
running build
running build_ext
cythoning src/cython/vapoursynth.pyx to build/temp.linux-x86_64-2.7/pyrex/vapoursynth.c
creating build/temp.linux-x86_64-2.7
creating build/temp.linux-x86_64-2.7/pyrex
building 'vapoursynth' extension
creating build/temp.linux-x86_64-2.7/build
creating build/temp.linux-x86_64-2.7/build/temp.linux-x86_64-2.7
creating build/temp.linux-x86_64-2.7/build/temp.linux-x86_64-2.7/pyrex
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I. -Isrc/cython -I/usr/include/python2.7 -c build/temp.linux-x86_64-2.7/pyrex/vapoursynth.c -o build/temp.linux-x86_64-2.7/build/temp.linux-x86_64-2.7/pyrex/vapoursynth.o
build/temp.linux-x86_64-2.7/pyrex/vapoursynth.c: In function ‘__pyx_f_11vapoursynth_createFormat’:
build/temp.linux-x86_64-2.7/pyrex/vapoursynth.c:6181:13: warning: assignment discards ‘const’ qualifier from pointer target type [enabled by default]
build/temp.linux-x86_64-2.7/pyrex/vapoursynth.c: In function ‘__pyx_pf_11vapoursynth_4Core_16version’:
build/temp.linux-x86_64-2.7/pyrex/vapoursynth.c:13004:13: warning: assignment discards ‘const’ qualifier from pointer target type [enabled by default]
build/temp.linux-x86_64-2.7/pyrex/vapoursynth.c: In function ‘vpy_clearVariable’:
build/temp.linux-x86_64-2.7/pyrex/vapoursynth.c:19060:3: warning: ‘__pyx_r’ may be used uninitialized in this function [-Wmaybe-uninitialized]
build/temp.linux-x86_64-2.7/pyrex/vapoursynth.c: In function ‘vpy_getOutput’:
build/temp.linux-x86_64-2.7/pyrex/vapoursynth.c:18074:3: warning: ‘__pyx_r’ may be used uninitialized in this function [-Wmaybe-uninitialized]
build/temp.linux-x86_64-2.7/pyrex/vapoursynth.c: In function ‘vpy_getCore’:
build/temp.linux-x86_64-2.7/pyrex/vapoursynth.c:18388:3: warning: ‘__pyx_r’ may be used uninitialized in this function [-Wmaybe-uninitialized]
build/temp.linux-x86_64-2.7/pyrex/vapoursynth.c: In function ‘vpy_evaluateScript’:
build/temp.linux-x86_64-2.7/pyrex/vapoursynth.c:16885:3: warning: ‘__pyx_r’ may be used uninitialized in this function [-Wmaybe-uninitialized]
build/temp.linux-x86_64-2.7/pyrex/vapoursynth.c: In function ‘vpy_evaluateFile’:
build/temp.linux-x86_64-2.7/pyrex/vapoursynth.c:17430:3: warning: ‘__pyx_r’ may be used uninitialized in this function [-Wmaybe-uninitialized]
build/temp.linux-x86_64-2.7/pyrex/vapoursynth.c: In function ‘vpy_getVariable’:
build/temp.linux-x86_64-2.7/pyrex/vapoursynth.c:18668:3: warning: ‘__pyx_r’ may be used uninitialized in this function [-Wmaybe-uninitialized]
creating build/lib.linux-x86_64-2.7
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/build/temp.linux-x86_64-2.7/pyrex/vapoursynth.o -L. -Lbuild -lvapoursynth -o build/lib.linux-x86_64-2.7/vapoursynth.so
running install_lib
copying build/lib.linux-x86_64-2.7/vapoursynth.so -> /usr/lib/python2.7/site-packages
running install_egg_info
Removing /usr/lib/python2.7/site-packages/VapourSynth-1.0.0-py2.7.egg-info
Writing /usr/lib/python2.7/site-packages/VapourSynth-1.0.0-py2.7.egg-info

[qyot27]

[Install FFmpeg, make sure to use --enable-pic for 64-bit; the guide on FFmpeg's wiki is enough]

EDIT: Now updated for Ubuntu 14.04.

Installing the prerequisites:
sudo apt-get install python3-dev wget python3-sphinx cython3

Building VapourSynth itself:
git clone git://github.com/vapoursynth/vapoursynth.git
cd vapoursynth
./bootstrap.py
./waf configure
./waf build
sudo checkinstall --pkgname=vapoursynth --pkgversion="$(grep Version build/pc/vapoursynth.pc | \
sed 's/ /"/g' | cut -f2 -d "\"")"R-r"$(git rev-list --count HEAD)" \
--backup=no --deldoc=yes --delspec=yes --deldesc=yes --strip=yes \
--stripso=yes --addso=yes --fstrans=no --default ./waf install

sudo checkinstall --pkgname=vapoursynth-cython --pkgversion="$(grep Version build/pc/vapoursynth.pc | \
sed 's/ /"/g' | cut -f2 -d "\"")"R-r"$(git rev-list --count HEAD)" \
--backup=no --deldoc=yes --delspec=yes --deldesc=yes --strip=yes \
--stripso=yes --addso=yes --fstrans=no --default --requires=vapoursynth \
python3 ./setup.py install

Building FFMS2:
git clone git://github.com/FFMS/ffms2.git
cd ffms2
./configure --enable-shared --with-pic
make
sudo checkinstall --pkgname=ffms2 --pkgversion="1:$(./version.sh)-git" \
--backup=no --deldoc=yes --delspec=yes --deldesc=yes --strip=yes \
--stripso=yes --addso=yes --fstrans=no --default

[lanzorg]

I actually use an Avisynth script called FineSharp with FFDshow for my HD sources on a Windows machine.

But I switched to Archlinux last year for web development purpose, now I can't work on Windows anymore.

I see that FineSharp was ported on Vapoursynth and my question is can I use it on Archlinux like I use Avisynth on my Windows machine.
For example with MPV (mplayer2 with some improvements)? If yes can you explain me or give me useful links about it?

Thank you in advance.

[jackoneill]

I see that FineSharp was ported on Vapoursynth and my question is can I use it on Archlinux like I use Avisynth on my Windows machine.
For example with MPV (mplayer2 with some improvements)? If yes can you explain me or give me useful links about it?

Thank you in advance.

Not exactly. If finesharp doesn't require any Avisynth filters, you can process your videos in Arch Linux, but not while you watch them with mpv. mpv doesn't support this. I'm not aware of any Linux video player that can do this.

[lanzorg]

OK thanks for the answer, so any video player on any platform can't use vapoursynth (for the moment).

I really need FineSharp, the mpv lavfi unsharp filter is crap as hell.

So I must use something like VirtualDub to pre-encode my video like I did with Avisynth 6 years ago. Or it is CLI only? :(
Do you have some tuts about it on Linux?

[mavrick_264]

Hi,
i am a newbee here and also as far as X264 is concerned. For now i have been given a task to run the DirectX264 code in MSVS 2010, Ultimate.
However, i have come across the errors:
1. Error 444 error C2440: 'initializing' : cannot convert from 'AVS_Value' to 'short'
Path: ...\X264_codes\For_MSVS_from_svn\X264_on_VS\input\avs.c 223
2. Error 446 error C2440: 'initializing' : cannot convert from 'AVS_Value' to 'void *'
Path: ...\X264_codes\For_MSVS_from_svn\X264_on_VS\input\avs.c 223

What can be the solution to this?
Here AVS_Value is a stucture as:
typedef struct AVS_Value AVS_Value;
struct AVS_Value {
short type; // 'a'rray, 'c'lip, 'b'ool, 'i'nt, 'f'loat, 's'tring, 'v'oid, or 'l'ong
// for some function e'rror
short array_size;
union {
void * clip; // do not use directly, use avs_take_clip
char boolean;
int integer;
float floating_pt;
const char * string;
const AVS_Value * array;
} d;
};

it has made way to too confused.
These are the lines where the errors are occuring:
Line 223 : AVS_Value arg_arr[] = { res, avs_new_value_bool( info->interlaced ), avs_new_value_string( matrix ) };
Line 234 : AVS_Value arg_arr[] = { res, avs_new_value_string( levels ) };

Can anybody pls help?
Thanks.
Regards,
Mavrick_264

[jackoneill]

OK thanks for the answer, so any video player on any platform can't use vapoursynth (for the moment).

I really need FineSharp, the mpv lavfi unsharp filter is crap as hell.

So I must use something like VirtualDub to pre-encode my video like I did with Avisynth 6 years ago. Or it is CLI only? :(
Do you have some tuts about it on Linux?


$ vspipe moo.py - -y4m | x264 --stuff -o moo.mkv -

Then you need to mux the audio/subtitles too...

[Chikuzen]

OK thanks for the answer, so any video player on any platform can't use vapoursynth (for the moment).
on Windows, you can play your vpy on almost all players via VSFS.

I really need FineSharp, the mpv lavfi unsharp filter is crap as hell.
Regrettably, finesharp requires removegrain.dll with this time.
It cannot be used on Linux until someone ports removegrain to VS.

[sneaker_ger]

Also note that VapourSynth is video only - no audio. Also, the output methods are usually CFR only, making usage for playback even more complicated.

[Tima]

I experience some strange crash with RemoveGrain plugin when trying to run VS script, which uses QTGMC.

Crash report from VirtualDub: http://pastebin.com/YRZbrDBg

My script:

src=r'n01-01.avi'
plugins_vs=r'C:\\Bin\\VapourSynth\\filters\\'
plugins_avs_megui=r'C:\\Bin\\MeGUI\\tools\\avisynth_plugin\\'
plugins_avs_custom=r'C:\\Bin\\avisynth_plugins_custom\\'

import vapoursynth as vs
import sys
import havsfunc
core = vs.get_core(threads = 1)
haf = havsfunc.HAvsFunc(core)


core.std.LoadPlugin(plugins_vs + r'ffms2.dll')
core.std.LoadPlugin(plugins_vs + r'fmtconv.dll')
#core.std.LoadPlugin(plugins_vs + r'd2vsource_beta3_mingw.dll')
core.std.LoadPlugin(plugins_vs + r'nnedi3.dll')
core.std.LoadPlugin(plugins_vs + r'eedi3.dll')
core.std.LoadPlugin(plugins_vs + r'scenechange.dll')
core.std.LoadPlugin(plugins_vs + r'temporalsoften2.dll')
core.std.LoadPlugin(plugins_vs + r'genericfilters.dll')

core.avs.LoadPlugin(plugins_vs + r'avisynthfilters.dll')

core.avs.LoadPlugin(plugins_avs_megui + r'EEDI2.dll')
core.avs.LoadPlugin(plugins_avs_megui + r'TDeint.dll')

core.avs.LoadPlugin(plugins_avs_custom + r'AddGrainC.dll')
core.avs.LoadPlugin(plugins_avs_custom + r'dfttest.dll')
core.avs.LoadPlugin(plugins_avs_custom + r'FFT3DFilter.dll')
core.avs.LoadPlugin(plugins_avs_custom + r'mvtools2.dll')
core.avs.LoadPlugin(plugins_avs_custom + r'nnedi.dll')
core.avs.LoadPlugin(plugins_avs_custom + r'nnedi2.dll')
core.avs.LoadPlugin(plugins_avs_custom + r'RemoveGrainSSE2.dll')
core.avs.LoadPlugin(plugins_avs_custom + r'RepairSSE2.dll')
core.avs.LoadPlugin(plugins_avs_custom + r'VerticalCleanerSSE2.dll')

core.avs.LoadPlugin(plugins_avs_custom + r'mt_masktools-26.dll')


clip = core.ffms2.Source(src)
clip = haf.QTGMC(clip, Preset='Slower', TFF=True)
clip = core.resize.Bicubic(clip,640,480)

clip.set_output()

[Reel.Deel]

Hello,

I'm trying to pipe RGB48 to ImageMagick in order to create a 16-bit PNG but I'm not having much success.
This is the script I'm using:
import vapoursynth as vs
core = vs.get_core()
core.std.LoadPlugin(path=r'C:/vsimagereader.dll')

# 16-bit 720X1280 PNG
src = core.imgr.Read(['C:/IMG_0577_720.png'])

# Resize
src = core.fmtc.resample(clip=src, w=360, h=640)

# Output
src.set_output()

The command line I'm using:
"vspipe.exe" "RGB48 Test.vpy" - | convert -depth 16 -size 360x640 rgb:- "RGB48 Test.png"

The result should look like this (https://www.dropbox.com/s/r5gw5k6lwhhinjh/RGB48%20Test%208.png), instead I get this (https://www.dropbox.com/s/kj1pqiw52669sqg/RGB48%20Test%208%202.png).

In the quest for answers I read this thread (http://www.imagemagick.org/discourse-server/viewtopic.php?f=1&t=18943) but I still can't figure it out.
Any help will be greatly appreciated. :)

[Myrsloik]

The pictures seem to be broken but my guess is that it expects packed rgb, vs outputs planar rgb.

[Reel.Deel]

I fixed the links. Is there any way to change planar to packed within VS?
If not then I guess I'll create an account over IM to ask if there's a way for IM to correctly accept planar RGB.
Thanks for the help regardless.

[Myrsloik]

I fixed the links. Is there any way to change planar to packed within VS?
If not then I guess I'll create an account over IM to ask if there's a way for IM to correctly accept planar RGB.
Thanks for the help regardless.

No, I try to keep everything planar as much as possible. The easiest way would probably be to write a small program that converts planar rgb48 to packed when it's piped through it.

Btw, the links are still broken. I don't think the files are public in your dropbox.

[Reel.Deel]

Fair enough. If I find a solution in ImageMagick I'll post it here. Thanks again.

Ok, this time I really fixed the links :o.
Third times a charm :).

[SAPikachu]

Hello,

I'm trying to pipe RGB48 to ImageMagick in order to create a 16-bit PNG but I'm not having much success.
This is the script I'm using:
import vapoursynth as vs
core = vs.get_core()
core.std.LoadPlugin(path=r'C:/vsimagereader.dll')

# 16-bit 720X1280 PNG
src = core.imgr.Read(['C:/IMG_0577_720.png'])

# Resize
src = core.fmtc.resample(clip=src, w=360, h=640)

# Output
src.set_output()

The command line I'm using:
"vspipe.exe" "RGB48 Test.vpy" - | convert -depth 16 -size 360x640 rgb:- "RGB48 Test.png"

The result should look like this (https://www.dropbox.com/s/r5gw5k6lwhhinjh/RGB48%20Test%208.png), instead I get this (https://www.dropbox.com/s/kj1pqiw52669sqg/RGB48%20Test%208%202.png).

In the quest for answers I read this thread (http://www.imagemagick.org/discourse-server/viewtopic.php?f=1&t=18943) but I still can't figure it out.
Any help will be greatly appreciated. :)

You can emulate packed RGBA with GRAY output. Here is a snippet I just tried:

r = core.std.ShufflePlanes([src], [0], vs.GRAY)
g = core.std.ShufflePlanes([src], [1], vs.GRAY)
b = core.std.ShufflePlanes([src], [2], vs.GRAY)
a = core.std.BlankClip(clip=r, color=[65535])

last = core.std.Interleave([r,b,g,a])# r b g a -> rb ga -> rgba

# Turn right
last = core.std.Transpose(last)
# last = core.std.FlipHorizontal(last) # EDIT: Not needed

last = core.std.DoubleWeave(last, True)
last = core.std.SelectEvery(last, 2, 0)
last = core.std.DoubleWeave(last, True)
last = core.std.SelectEvery(last, 2, 0)

# Turn left
last = core.std.Transpose(last)
# last = core.std.FlipVertical(last) # EDIT: Not needed

last.set_output()



Tested conversion with the following command (note that we have to use rgba instead of rgb):

"vspipe.exe" test.vpy - | convert -depth 16 -size 1920x1080 rgba:- test.png

[Chikuzen]

@SAPikachu
I think that FlipHorizontal/FlipVertical is unneeded.
Just Transpose x 2 is enough.

[SAPikachu]

@SAPikachu
I think that FlipHorizontal/FlipVertical is unneeded.
Just Transpose x 2 is enugh.

Yes they are indeed unneeded. I followed advice of Vapoursynth documentation and didn't notice that. Just updated the snippet. Thanks for pointing out this.

[Myrsloik]

Welcome to the weekly "the project isn't dead" announcement.

Due to wanting to fix some issues properly the x64 builds will be delayed a bit.

Note that the current and past 4 git revisions are broken.

[Reel.Deel]

@SAPikachu

I've been out of town so I just now tried your solution.
It works as expected. Thank you very much.

[aegisofrime]

Hi!

Sorry if I'm asking in the wrong section. I'm wondering, how do I replicate the rpow2 function of NNEDI3? As far as I understand the VS version of NNEDI3 lacks the rpow2 function. Thanks!

[jackoneill]

Hi!

Sorry if I'm asking in the wrong section. I'm wondering, how do I replicate the rpow2 function of NNEDI3? As far as I understand the VS version of NNEDI3 lacks the rpow2 function. Thanks!

Yes, it belongs here: http://forum.doom9.org/showthread.php?t=166434.

I'm fairly confident I can add that functionality.

[aegisofrime]

Yes, it belongs here: http://forum.doom9.org/showthread.php?t=166434.

I'm fairly confident I can add that functionality.

I searched through that topic but no one asked about rpow2 :p

Thanks, looking forward to it!

[Myrsloik]

Here's RC1 for R20 (https://dl.dropboxusercontent.com/u/73468194/vapoursynth_r20_rc1.exe).

Report any bugs you find here. I'll post some FrameEval examples later.

Changes:
r20:
rewrote the ugliest threading code, the locking is now a bit more fine grained in general
added frameeval, a function to allow per frame filter evaluation
blankclip now generates a new frame on every request to keep memory usage down, added the keep argument to optionally always return the same frame
the message handler now takes an extra user data pointer, the handler and data pointer is still per process instead of per core though
added plugin autoloading for linux (jackoneill)
added plugin autoloading for windows, autoloaded plugins for all users go into <installdir>\plugins and per user autoloaded plugins go into <appdata>\vapoursynth\plugins
include the python backtrace in errors when available
fixed a bug in the propagation of filter errors, requesting a frame with an error twice no longer makes vapoursynth hang
fixed the error that happens when avisource is used to open a vs script inside a vs script
added override support to vdecimate (nodame)
added -version option to vspipe and other small fixes
addborders now properly rejects clips where the colorspace can change
addborders and blankclip now properly default to black for all colorspaces
added tdeint propery to the prefetch list_functions
fixed a cache bug that would make caches adapt too slowly/not at all
only output the first 200 slow warnings per avisynth filter
don't prefetch any frames by default for avisynth filters

[Tima]

Anyone else has RemoveGrain crash when trying to use QTGMC? :)

[the same was with r19 too]

[Are_]

Long time no testing due to various things and now non of my scripts works :/

I can easily make VS crash with a simple Repari call (0.9 and 1.0).

This script for instance:
import vapoursynth as vs
core = vs.get_core()

core.avs.LoadPlugin(path=r'DGDecodeNV.dll')
core.avs.LoadPlugin(path=r'RepairSSE2.dll')
core.avs.LoadPlugin(path=r'RemoveGrainSSE2.dll')

src = core.avs.DGSource('01.dgi')

main = core.avs.RemoveGrain(src, 1)
main = core.avs.Repair(main, src, 17)

main.set_output()

[Reel.Deel]

Just tested with the script Are_ posted, I can confirm the crash with R20 RC1.
Here's the log (https://www.dropbox.com/s/2c7zswoxrob79d0/crashinfo.txt) from VDub.

[Myrsloik]

Just tested with the script Are_ posted, I can confirm the crash with R20 RC1.
Here's the log (https://www.dropbox.com/s/2c7zswoxrob79d0/crashinfo.txt) from VDub.

Time to look into it. Let's see what evils these plugins do...

[Myrsloik]

Here's R20 RC2 (https://dl.dropboxusercontent.com/u/73468194/vapoursynth_r20_rc2.exe).

It fixes the avisynth compatibility for filters without prefetching. I also added removegrain and repair to the prefetch list since apparently I forgot them.

[Tima]

RC2 still crashes with RemoveGrain.

VirtualDub log (http://pastebin.com/08RYUaNU)

[Myrsloik]

RC2 still crashes with RemoveGrain.

VirtualDub log (http://pastebin.com/08RYUaNU)

I used the same script Are_ posted (but with blankclip as the source). Does that one crash for you as well now?

If it still crashes, can you send me the exact repair and removegrain dlls you are using?

There are simply too many variations of them for me to find and try all. It's also interesting because it now crashes inside removegrain code and not in vapoursynth like like in reel.deel's log.

[Tima]

I used the same script Are_ posted (but with blankclip as the source). Does that one crash for you as well now?

If it still crashes, can you send me the exact repair and removegrain dlls you are using?


I used my own script, similar to Are_'s. It crashes for me with any RemoveGrain version I tried.

src=r'video-raw\\n01-01.avi'
plugins_avs_custom=r'C:\\Bin\\avisynth_plugins_custom\\'

import vapoursynth as vs
import sys
core = vs.get_core(threads = 1)

core.avs.LoadPlugin(plugins_avs_custom + r'RemoveGrainSSE2.dll')

clip = core.ffms2.Source(src)
clip = core.avs.RemoveGrain(clip, 1)

clip.set_output()

RemoveGrain DLL (correct link) (http://rghost.net/49213483)

[Myrsloik]

I used my own script, similar to Are_'s. It crashes for me with any RemoveGrain version I tried.

src=r'video-raw\\n01-01.avi'
plugins_avs_custom=r'C:\\Bin\\avisynth_plugins_custom\\'

import vapoursynth as vs
import sys
core = vs.get_core(threads = 1)

core.avs.LoadPlugin(plugins_avs_custom + r'RemoveGrainSSE2.dll')

clip = core.ffms2.Source(src)
clip = core.avs.RemoveGrain(clip, 1)

clip.set_output()

RemoveGrain DLL (http://rghost.net/49202817)

You uploaded the wrong dll. Try again.

[Tima]

You uploaded the wrong dll. Try again.

Oops, fixed :)

[Myrsloik]

Oops, fixed :)

Odd. That file works on my computer. Head scratching time.

Your cpu does have sse2 support, right?

[Tima]

Odd. That file works on my computer. Head scratching time.

Your cpu does have sse2 support, right?

Yes (BTW it crashes with any RemoveGrain version, including non-SSE).

(relevant system specs: Core i7 940, Win8 x64)

[Myrsloik]

Yes (BTW it crashes with any RemoveGrain version, including non-SSE).

(relevant system specs: Core i7 940, Win8 x64)

Can you run the script with vspipe and see if it prints any error?

Do you have the source code for that version of removegrain or know where i can find it?

[Tima]

Can you run the script with vspipe and see if it prints any error?
No errors in console (just an std warning there is no user-specific plugin auto-loading folder), vspipe just crashes :)

I ran it 10 times and got these errors:
Unhandled exception at 0x0288F6C0 in vspipe.exe: 0xC0000005: Access violation executing location 0x0288F6C0.
Unhandled exception at 0x726F486B in vspipe.exe: 0xC0000005: Access violation executing location 0x726F486B.
Unhandled exception at 0x726F486B in vspipe.exe: 0xC0000005: Access violation executing location 0x726F486B.
Unhandled exception at 0x726F486B in vspipe.exe: 0xC0000005: Access violation executing location 0x726F486B.
Unhandled exception at 0x029A2FB8 in vspipe.exe: 0xC0000005: Access violation executing location 0x029A2FB8.
Unhandled exception at 0x55446D56 (RemoveGrainSSE2.dll) in vspipe.exe: 0xC0000005: Access violation reading location 0x00000014.
Unhandled exception at 0x726F486B in vspipe.exe: 0xC0000005: Access violation executing location 0x726F486B.
Unhandled exception at 0x696C436B in vspipe.exe: 0xC0000005: Access violation executing location 0x696C436B.
Unhandled exception at 0x55446D56 (RemoveGrainSSE2.dll) in vspipe.exe: 0xC0000005: Access violation reading location 0x00000014.
Unhandled exception at 0x726F486B in vspipe.exe: 0xC0000005: Access violation executing location 0x726F486B.

Stack trace hints that RemoveGrainSSE2.dll is involved in every case.

Could you build debug version (with pdb) for me to test?

The version Tima uses is Vit's Modded Plugins (http://forum.doom9.org/showthread.php?t=156028)(source code included in the package).
I wonder whether he tried the original version instead of the modded version.
Correct. I tried all dlls from all three Vit's packages -- each of them crashes.
Stable v0.9 from http://avisynth.nl/index.php/RemoveGrain crashes too.

[Myrsloik]

Here are two compiles of the vit removegrain version (https://dl.dropboxusercontent.com/u/73468194/rg.7z).

Try the release one first and see if it crashes. There's a pdb included for the debug version.

[Tima]

Try the release one first and see if it crashes. There's a pdb included for the debug version.

Both r20-rc1 and r20-rc2 with both builds of RG crash in similar ways:
Unhandled exception at 0x726F486B in vspipe.exe: 0xC0000005: Access violation executing location 0x726F486B.
Unhandled exception at 0x6BEC8DFB (RemoveGrain.dll) in vspipe.exe: 0xC0000005: Access violation reading location 0x00000014.

Stack trace is the same in all cases:
> 726f486b() Unknown
[Frames below may be incorrect and/or missing]
RemoveGrain.dll!RemoveGrain::RemoveGrain(PClip clip, int * mode, bool planar) Line 7565 C++
RemoveGrain.dll!CreateRemoveGrain(AVSValue args, void * user_data, IScriptEnvironment * env) Line 7628 C++
vapoursynth.dll!5bc57a91() Unknown
vapoursynth.dll!5bc44553() Unknown
vapoursynth.dll!5bc47dd3() Unknown
vapoursynth.pyd!5cdef90c() Unknown
vapoursynth.pyd!5cdf24ff() Unknown
python33.dll!1e07bd5b() Unknown
python33.dll!1e0f887c() Unknown
python33.dll!1e0f8ec1() Unknown
python33.dll!1e0faf2d() Unknown
python33.dll!1e0fbf7d() Unknown
python33.dll!1e0fc0e4() Unknown
vapoursynth.pyd!5cdd576a() Unknown
vapoursynth.pyd!5cde039d() Unknown
vapoursynth.pyd!5cde6931() Unknown
vsscript.dll!64fc141b() Unknown
vspipe.exe!000a2360() Unknown
ntdll.dll!77b6dd84() Unknown
ntdll.dll!77b973bc() Unknown
msvcr100.dll!6caf0949() Unknown
vspipe.exe!000a2896() Unknown
vspipe.exe!000a2919() Unknown
vspipe.exe!000a3377() Unknown
msvcr100.dll!6caf263d() Unknown
vspipe.exe!000a2a90() Unknown
kernel32.dll!7528850d() Unknown
ntdll.dll!77b9bf39() Unknown
ntdll.dll!77b9bf0c() Unknown

[Myrsloik]

The removegrain/repair issue has now been debugged. Basically removegrain/repair are horribly broken and will never work well.

I'll try to get a proper port of removegrain done quickly so QTGMC remains usable.

[Myrsloik]

Here's R20 RC3 (https://dl.dropboxusercontent.com/u/73468194/vapoursynth_r20_rc3.exe). This will probably be the final released version.

Changes from RC2:
Checks the input colorspace to avisynth filters, prevents odd crashes under certain conditions
Doesn't keep copies of all filter arguments around

Note that RemoveGrain/Repair violate the avisynth api so they'll never be supported. It's even explicitly stated in the avisynth documentation to not store a pointer to the IScriptEnvironment in a global variable but apparently some people don't read much.

[Mystery Keeper]

I've got WinPython 3.3 installed and its internal python directory in my PATH variable. And installer still says I haven't got python installed.

[the_weirdo]

I've got WinPython 3.3 installed and its internal python directory in my PATH variable. And installer still says I haven't got python installed.

Is your WinPython 32-bit or 64-bit? Vapoursynth currently need 32-bit Python, I think.

[Mystery Keeper]

Solved. Had to open WinPython control panel and "Register distribution".

[Myrsloik]

I've released R20 and have made the usual blog post summary (http://www.vapoursynth.com/2013/10/r20-closer-to-done/). I also ditched the annoying adfly advertisement stuff. Full changelog in the first post in this thread. The release is identical to RC3.

In case anyone's wondering I made like $3.5 in a year...

[Myrsloik]

https://dl.dropboxusercontent.com/u/73468194/error.png
This is in no way inspired by avisynth. Just saying.

[sl1pkn07]

Myrsloik, removegrain/repair port is ready to use/test? not added in a waf build script (i add by myself, but get a lots of error http://sl1pkn07.no-ip.com/paste/view/21e988df (i use this wscript patch: http://sl1pkn07.no-ip.com/paste/view/4e3ff441. but i not sure if is correct))

greetings

edit: the auto [url.][/url.] is broken?

[Myrsloik]

Myrsloik, removegrain/repair port is ready to use/test? not added in a waf build script (i add by myself, but get a lots of error http://sl1pkn07.no-ip.com/paste/view/21e988df (i use this wscript patch: http://sl1pkn07.no-ip.com/paste/view/4e3ff441. but i not sure if is correct))

greetings

It's not added yet because there are some very small non-windows issues. I'll probably fix the last lines tomorrow.

[sl1pkn07]

oks, thanks

[LaTo]

I've released R20 and have made the usual blog post summary (http://www.vapoursynth.com/2013/10/r20-closer-to-done/). I also ditched the annoying adfly advertisement stuff. Full changelog in the first post in this thread. The release is identical to RC3.

In case anyone's wondering I made like $3.5 in a year...

I just read your blog post...
And I'm using a build-in font in my AVS plugins, which is more or less the same principle as vps-scrawl but with alpha-blending.
It is slower to draw but it is nice looking (more modern style).
Let me know if you're interested or not, if so I will post it here...

[Myrsloik]

I just read your blog post...
And I'm using a build-in font in my AVS plugins, which is more or less the same principle as vps-scrawl but with alpha-blending.
It is slower to draw but it is nice looking (more modern style).
Let me know if you're interested or not, if so I will post it here...

Why not make it a plugin so the users can decide? And is the font you're using under a reasonable license?

In my opinion the scrawl features are enough since it's only for debug printing/simple labeling and so on. If you want something better you mas as well go libass or some other proper subtitle renderer.

[jackoneill]

I just read your blog post...
And I'm using a build-in font in my AVS plugins, which is more or less the same principle as vps-scrawl but with alpha-blending.
It is slower to draw but it is nice looking (more modern style).
Let me know if you're interested or not, if so I will post it here...

What does it look like?

[Reel.Deel]

Maybe like the font used in AutoGain's (http://forum.doom9.org/showthread.php?t=167573) or SmoothAdjust's (http://forum.doom9.org/showthread.php?t=154971) elegant debug view mode?

http://latoninf.free.fr/d9/AG/AutoGain_DebugView_thumb.png (http://latoninf.free.fr/d9/AG/AutoGain_DebugView.png)
* Click the image to enlarge *

http://latoninf.free.fr/d9/SA/SCshow_mini.png (http://latoninf.free.fr/d9/SA/SCshow.png)
* SmoothCurve - Click the image to enlarge *

[jackoneill]

Maybe like the font used in AutoGain's (http://forum.doom9.org/showthread.php?t=167573) or SmoothAdjust's (http://forum.doom9.org/showthread.php?t=154971) elegant debug view mode?


Heh, that looks like DejaVu Sans Mono (which isn't a bitmap font, I think).

[LaTo]

AvsFont.7z (http://latoninf.free.fr/d9/AvsFont.7z)

The .7z file contains only a header with the raw data and a small example (pseudo-code, not tested).

Unfortunately I do not have time to do a more complete package,
so you will have to create your classes and functions yourself.

[sneaker_ger]

I got a strange issue running R20 and Chikuzen's ImageReader 0.2.1. I'm trying to open this image (http://www.parabolaresearch.com/mona_lisa_original.jpg). When using the VfW interface the application simply closes immediately, no error message or anything (tested VirtualDub and MPC-HC). vspipe shows the correct info but if you try to pipe it shows:
No frame returned at the end of processing by
QWaitCondition: Destroyed while threads are still waiting

[Myrsloik]

I got a strange issue running R20 and Chikuzen's ImageReader 0.2.1. I'm trying to open this image (http://www.parabolaresearch.com/mona_lisa_original.jpg). When using the VfW interface the application simply closes immediately, no error message or anything (tested VirtualDub and MPC-HC). vspipe shows the correct info but if you try to pipe it shows:
No frame returned at the end of processing by
QWaitCondition: Destroyed while threads are still waiting

That's odd. Will test it tomorrow. It should at least print the name of the offending filter.

[Myrsloik]

I got a strange issue running R20 and Chikuzen's ImageReader 0.2.1. I'm trying to open this image (http://www.parabolaresearch.com/mona_lisa_original.jpg). When using the VfW interface the application simply closes immediately, no error message or anything (tested VirtualDub and MPC-HC). vspipe shows the correct info but if you try to pipe it shows:
No frame returned at the end of processing by
QWaitCondition: Destroyed while threads are still waiting

The issue is that vsimagereader accesses a NULL pointer. I think Chikuzen will post an updated version later today.

[Myrsloik]

Here's a test version (https://dl.dropboxusercontent.com/u/73468194/vapoursynth_r21_test1.exe) with many changes and improvements.

Notable changes since R20:

now properly flips compatrgb32 when converting, previously conversions to/from would make the picture upside down
optimized 8 bit merge and maskedmerge
fixed crash bugs in planeaverage/difference/cliptoprop/proptoclip introduced in r20
now flattens frame properties as well, this is the proper documented behavior
merged all selectclip functionality into frameeval, selectclip was removed as a separate function
vsvfw now returns a clip with the error message if there's an error
added a basic text printing filter to the core, code written by nodame
it's now possible to use dir() on frame.props and core.namespace in python to list interesting properties and attributes
added a compiled version of assvapour, it's probably a bit broken when it comes to font selection at the moment
ported removegrain/repair based on dither tools' code, support 8-16 bit input
fixed a cache issues where it would shrink to far too small sizes
fixed several bugs in shuffleplanes, now it will correctly deduce the dimensions of the output in more cases and properly reject variable size/format input
fixed a rare hang that could happen in avisynth compatibility when only using one thread
adjusted avisynth compat prefetching lists, specifically MCompensate should be faster now

Help test the changes. This build should be better than R20 in every way.

[gpower2]

Will we see any speed difference with this version, compared to a traditional AviSynth script?

Especially in true multicore processors...

[aegisofrime]

Hi,

I seem to be getting strange artifacts with QTGMC on r21 that wasn't present on r19 (never tried r20):

Good (r19):

http://i.imgur.com/5bbXG3wl.jpg (http://i.imgur.com/5bbXG3w.jpg)

Bad (r21):

http://i.imgur.com/ZKK1JPLl.jpg (http://i.imgur.com/ZKK1JPL.jpg)

Note: The artifacts aren't as obvious on the thumbnail, click on thumbnail for full size image in which the artifacts are more obvious.

Here's my script that I used for r21, just in case it's a screwup on my end (which is probably very likely)


import vapoursynth as vs
import havsfunc
import sys
core = vs.get_core(threads=4)
haf = havsfunc.HAvsFunc(core)
core.std.LoadPlugin(r'C:\Program Files (x86)\VapourSynth\filters\temporalsoften.dll')
core.std.LoadPlugin(r'C:\Program Files (x86)\VapourSynth\filters\libnnedi3.dll')
core.std.LoadPlugin(r'C:\Program Files (x86)\VapourSynth\filters\fmtconv.dll')
core.std.LoadPlugin(r'C:\Program Files (x86)\VapourSynth\filters\scenechange.dll')
core.std.LoadPlugin(r'C:\Program Files (x86)\VapourSynth\filters\temporalsoften2.dll')
core.std.LoadPlugin(r'C:\Program Files (x86)\VapourSynth\filters\genericfilters.dll')
core.avs.LoadPlugin(r"C:\Program Files (x86)\AviSynth 2.5\plugins\DGDecode.dll")
core.avs.LoadPlugin(path=r'C:\Program Files (x86)\VapourSynth\filters\avisynthfilters.dll')
core.avs.LoadPlugin(path=r'C:\Program Files (x86)\AviSynth 2.5\plugins\mt_masktools-25.dll')
core.avs.LoadPlugin(path=r'C:\Program Files (x86)\AviSynth 2.5\plugins\mvtools2.dll')
core.avs.LoadPlugin(path=r'C:\Program Files (x86)\AviSynth 2.5\plugins\RemoveGrainSSE3.dll')
core.avs.LoadPlugin(path=r'C:\Program Files (x86)\AviSynth 2.5\plugins\FFT3DFilter.dll')
core.avs.LoadPlugin(path=r'C:\DGAVCIndex\DGAVCDecode.dll')
clip = core.avs.AVCSource(dga =r"I:\Test\Test.dga")
clip = haf.QTGMC(clip, Preset='Slow', SubPel=2,TFF=True)
clip.set_output(index = 0)


For r21 I commented out the two temporalsoften filters, moving the temporalfilter2.dll file to the autoload folder, since somehow if I load them manually it throws an error.

For r19 I am able to load them successfully without an issue.

Am I doing something wrong?

P.S: Apologies for the large image size as it was what I could get quickest!

[Myrsloik]

1. There's a bug in merge() in the test version. That's where the artifacts come from.
2. You get the error because you're probably loading the same thing several times.
3. Use the native vs removegrain/repair. The avisynth one is too broken to work.

I'll probably post a new test build tonight after I test all the new as properly.

[Myrsloik]

Here's a fixed test build (https://dl.dropboxusercontent.com/u/73468194/vapoursynth-r21-test2.exe). No other changes.

[Myrsloik]

http://i.imgur.com/a1nWNYM.png
Is cache size not allowed to be set >= 2048MB?

It is allowed. But there may be bugs related to it. I'll check and fix the code.

Note that vspipe.exe is flagged to support 3GB address space but even then you need to leave a bit as margin. VS only counts allocated frames towards the memory use and ignores the rest.

[Myrsloik]

Here's R21 RC1 (https://dl.dropboxusercontent.com/u/73468194/vapoursynth-r21-RC1.exe).

Changes since test 2:
fixed setting maximum memory use over 2GB from python
added an additional output format argument to modifyframe, also added checks to prevent crashes
now completely rejects adding known to be hopelessly broken functions from avisynth plugins (removegrain/repair/iscombed/colormatrix)

I will be going C++11/C99 for R22. There are simply too many useful things in there to skip it. It's also a long term (veeeeeery long term) path to not relying on Qt at all.

Don't forget to take a look at the bug tracker (https://github.com/vapoursynth/vapoursynth/issues?state=open) as well. There are some things like documentation and not too difficult programming tasks I need help with. Such improving the Expr filter (https://github.com/vapoursynth/vapoursynth/issues/3).

[Wilbert]

What's broken in ColorMatrix?

[Myrsloik]

What's broken in ColorMatrix?

It's only broken in VS. If I recall correctly it uses invoke internally to do fancy things and I'm not going to bother implementing that part of avisynth.

Likewise IsCombed is conceptually broken as well. You can run it just fine but you'll never be able to use its output.

[Mystery Keeper]

Thank you for a great work, Mysrloik. Could you please make an example of plugin working with different bit depths and both integer and real color component values?

[Myrsloik]

Thank you for a great work, Mysrloik. Could you please make an example of plugin working with different bit depths and both integer and real color component values?

Would extending the invert example (https://github.com/vapoursynth/vapoursynth/blob/master/sdk/invert_example.c) be enough for you or do you have a particular simple filtering operation in mind? The other place to look is at the existing internal filters. For example MaskedMerge (https://github.com/vapoursynth/vapoursynth/blob/master/src/core/simplefilters.c#L3491) is a very straightforward filter that support 8-16 bit int and 32 bit float.

For integer types there's really not much difference. Everything just has more significant bits.

Floating point is slightly different. There RGB and Y channels are in the 0 to 1 range. And UV/CoCg are -0.5 to 0.5 range.

[Mystery Keeper]

Ugh. Thank you for the hint. But IMO that source really needs to be broken into several smaller ones.

[Mystery Keeper]

Trying to port TempLinearApproximate to VapourSynth. So far it works well on Y plane, but produces weird result on U and V (plane=6). What am I doing wrong?
http://paste.org.ru/?tgifsl
Also, for some reason it is much slower than in AvySynth.

[Myrsloik]

Trying to port TempLinearApproximate to VapourSynth. So far it works well on Y plane, but produces weird result on U and V (plane=6). What am I doing wrong?
http://paste.org.ru/?tgifsl
Also, for some reason it is much slower than in AvySynth.

You're using the R21 RC, right? R20 has a cache bug that can make it very slow.

I'll compile and take a look at what exactly the code does. A bit later today...