Vapoursynth

[Yellow_]

Youka, thanks for the reply, I'm no coder so it isn't clear at all to me, I'm having to compile because I can find no PPA for Vapoursynth, so could you elaborate on how I add C99 mode? I google I find std=c99 but how do I set that for using Vapoursynths ./waf build ?

[Adub]

Actually, if you look up a few posts, you'll see Myrsloik's answer to this very problem.
linky (http://forum.doom9.org/showthread.php?p=1605209#post1605209)

[Yellow_]

Adub & Youka, thanks, so the 'answer' to my query was in fact:

svn checkout -r 365 http://vapoursynth.googlecode.com/svn/trunk/ vapoursynth-365

Or wait for a rectified commit.

-r 365 did the job compiling, now I run ./waf test and error:

ImportError: No module named vapoursynth

whereis vapoursynth, answer:

vapoursynth: /usr/local/lib/vapoursynth

whereis vapoursynth.so:

vapoursynth: /usr/local/lib/vapoursynth /usr/local/include/vapoursynth

libvapoursynth.so exisits in /usr/local/lib

[qyot27]

You have to run 'sudo ./setup.py install' after 'sudo ./waf install' to get the Cython module built and installed. Otherwise you just have the library and nothing that links it into Python.

Also note what I mentioned in my last post about the need for using Python 3.3 if you want yuv4mpeg pipes to work like they should. Ubuntu 11.10-12.10 ship with 3.2 if you grab the python3 package from the repositories (this finally changes with 13.04, which uses 3.3 for its python3 package). What you need are the python3.3 and python3.3-dev packages as well. For good measure you also probably want to compile Cython 0.17.1 or 0.17.2, although I'm not quite as certain on this one (again, 13.04 won't be affected by this; it has 0.17.2).

[Adub]

You're going to need to install Cython via pip in order to get the correct version, as the Ubuntu repositories have a much older version.

"pip install cython" should do it.

[sl1pkn07]

when build vapoursynth in archlinux (https://aur.archlinux.org/packages/vapoursynth-svn/) via makepkg

namcap (package checker) says:

sl1pkn07@sL1pKn07 vapoursynth-svn $ namcap vapoursynth-svn-383-1-x86_64.pkg.tar.xz
vapoursynth-svn W: ELF file ('usr/lib/libvapoursynth.so') has text relocations.
vapoursynth-svn W: ELF file ('usr/lib/libvapoursynth.so') has executable stack.

execstack: https://bugs.archlinux.org/task/26458
text relocation: https://bugs.archlinux.org/task/26434

my linker flags (makepkg.conf)

CFLAGS="-march=x86-64 -mtune=native -O2 -pipe -fstack-protector --param=ssp-buffer-size=4 -D_FORTIFY_SOURCE=2"
CXXFLAGS="-march=x86-64 -mtune=native -O2 -pipe -fstack-protector --param=ssp-buffer-size=4 -D_FORTIFY_SOURCE=2"

executable stack fix with: "execstack -c build/libvapoursynth.so" (adding "-Wa,--noexecstack" in CFLAGS don't work)
i don'k know fix text relocation issue

greetings

[Are_]

executable stack fix with: "execstack -c build/libvapoursynth.so" (adding "-Wa,--noexecstack" in CFLAGS don't work)

Adding -z,noexecstack to the LDFLAGS fixed it for me, I didn't try with the CFLAGS. (I'm on a different distro, but I'm almost sure your makepkg.conf is ignored, you have to manually modify wscript for custom C/LDFLAGS).

[sl1pkn07]

tnx man, work! without edit wscript, only and LDFLAGS+=",-z,noexecstack"

like this:


LDFLAGS+=",-z,noexecstack"
waf configure --prefix="/usr" --docs="true"
waf build $MAKEFLAGS
python setup.py build


but i think need "fix" in upstream

and no, makepkg.conf not ignore

greetings

[qyot27]

You're going to need to install Cython via pip in order to get the correct version, as the Ubuntu repositories have a much older version.

"pip install cython" should do it.
I also noticed that pipes stopped working right after installing python3-pip, so I'm not sure what's going on. It could be that what I'm seeing isn't a problem with pip (rather, it would be the python3-setuptools that it relies on), but with something else, like I describe below.

If there is something the matter with it there, installing Cython manually is really the only choice. It's just one line - sudo python3.3 setup.py install - after unpacking the tarball.



It seems that if I pipe from VS to MPV, it only works the first time. I usually exit MPV before playback stops of its own accord, since I'm really only testing if the pipe worked. Subsequent times it seems to fail, even with 3.3 installed (although again, on Windows it doesn't do this and pipes correctly every time, making me think there's something wrong with my python setup on Ubuntu - which is problematic, because aside from Cython and VapourSynth, it's repository-standard).

[Myrsloik]

If you haven't noticed it already all the compilation issues should be fixed now. The next release has a big pile of fixes and is coming soon but I've got one odd bug left to fix in vsfs before its ready. The vfw memory leak bug when reopening will most likely not get fixed in R17 because it turned out to be a very tricky one.

[sl1pkn07]

https://code.google.com/p/vapoursynth/source/detail?r=385

don't make ELF without execstack

[Myrsloik]

https://code.google.com/p/vapoursynth/source/detail?r=385

don't make ELF without execstack

I don't dabble much in the non-windows builds but a motivation would be nice.

[jackoneill]

https://code.google.com/p/vapoursynth/source/detail?r=385

don't make ELF without execstack
It's fixed in r388.

[sl1pkn07]

zankius

[Tima]

I have FFV1 avi file which hangs during processing when using stdout to x264, when being opened by avisource.AVISource(), but works fine with ffms2.Source() (all modes).

test.cmd:
python test.vpy | x264_64.exe --demuxer y4m --output "test.mp4" -
test.vpy:
import vapoursynth as vs
import sys
core = vs.Core()

core.std.LoadPlugin(r'C:\Bin\VapourSynth\filters\ffms2.dll')
core.std.LoadPlugin(r'C:\Bin\VapourSynth\filters\avisource.dll')

#clip = core.ffms2.Source('test.avi')
clip = core.avisource.AVISource('test.avi')

#last=clip

clip.output(sys.stdout, y4m=True)


test.avi:
http://rghost.net/42392962

[Chikuzen]

＠Myrsoik
How about adding _SceneChange to reserved frame property ?
If a source filter or an independent scene change detect filter set it to the clip, almost of all temporal filters will use it.
I think it is reasonable.

[Myrsloik]

＠Myrsoik
How about adding _SceneChange to reserved frame property ?
If a source filter or an independent scene change detect filter set it to the clip, almost of all temporal filters will use it.
I think it is reasonable.

Added to the documentation

[Myrsloik]

Here's a beta for you to try. The transpose filter is broken but otherwise it should work better (or equally bad) as r16 at least. Plenty of bug fixes and I just didn't have time to test it properly before my Christmas trip. Coding will continue the 27th so report any other issues you find that aren't on the bug tracker already.

VapourSynth r17 beta (https://dl.dropbox.com/u/73468194/vapoursynth_installer_r17_beta.exe)

[Revgen]

Thank you.

Have a Merry Christmas and enjoy your trip.

[sneaker_ger]

The installer complains about missing Python if Python was installed for the current user only.

Also, is installing a new version on top of the old one OK? I saw that the de-installer deletes some files after the restart, so you would have to restart at least once if you want to upgrade.

[Myrsloik]

The installer complains about missing Python if Python was installed for the current user only.

Also, is installing a new version on top of the old one OK? I saw that the de-installer deletes some files after the restart, so you would have to restart at least once if you want to upgrade.

That's only if the files are in use. As in you've got a script open while upgrading and that is very unlikely to work properly. The installer also simply runs the uninstaller first to try to clear out old stuff and only overwrites after that. I believe it should always work properly and won't bother looking at it unless you can actually make it break.

I'll try to fix the python detection at least for r17.

[Tima]

Vapoursynth doesn't work with latest stable ffms2 v2.17.

Latest beta builds from googlecode work, but index most of my *.avi (Lagarith+AC3) files incorrectly, and don't accept index files from v2.17.

[Myrsloik]

Vapoursynth doesn't work with latest stable ffms2 v2.17.

Latest beta builds from googlecode work, but index most of my *.avi (Lagarith+AC3) files incorrectly, and don't accept index files from v2.17.

2.17 is ancient and not really supported anymore. 2.18 would've been released if ffmpeg/libav weren't so busy breaking their own api.

Also, you'll have to elaborate on "doesn't work". And what is incorrect indexing to you anyway?

[StainlessS]

If ffms2 v2.17 is not recommended, what is, r722 (24K+ downloads) or r725 (2K+ downloads) ?
Thank you in advance.

[Myrsloik]

If ffms2 v2.17 is not recommended, what is, r722 (24K+ downloads) or r725 (2K+ downloads) ?
Thank you in advance.

It is what it says. The svn revisions r722 and r275 which are veeeeery different. 2.17 doesn't even compile with new ffmpeg/libav.

[StainlessS]

Scratches head, but no wiser. :confused:
Thank you anyway.

[Tima]

Also, you'll have to elaborate on "doesn't work".

I get this error when trying to open *.vpy script with ffms2.dll v2.17 in VirtualDub:
---------------------------
VirtualDub Error
---------------------------
Avisynth open failure:
Python exception: 'No entry point found in C:\\Bin\\VapourSynth\\filters\\ffms2.dll'
---------------------------
OK
---------------------------


With r722 I get error message about deprecated R1 API.

And what is incorrect indexing to you anyway?

With r725, ffmsindex.exe instantly makes 124-byte long index for one 2.2Gb file, and I get a crash in Vdub when opening corresponding *.vpy:

An out-of-bounds memory access (access violation) occurred in module 'ffms2'...
...reading address 00000004.

My source is interlaced FFV1+AC3 *.avi file, captured by vdub.
Seems it has something to do with file length, because after trimming and direct stream copy to another (smaller) file, it indexes fine.

Update:
It definitely has something to do with 2Gb file size boundary.
I stripped audio from my source and trimmed down remaining video several times to get various file sizes.
r725 start indexing nicely right after going under 2Gb file size.

Update 2: The issue is no longer present in new ffms2 build (http://forum.doom9.org/showthread.php?p=1607732).

[Myrsloik]

I've released R17 which is hugely improved in most ways. Mostly in that I've tracked down the bugs in avisource which was released a bit before it was ready. It also has a new subtitle filter based on libass contributed by lachs0r.

As usual a slightly longer blog post can be found here (http://www.vapoursynth.com/2012/12/r17-a-soft-package/).

MVTools is next on my list of filters to attack but this will be one long project...

[Chikuzen]

I would appreciate some help in implementing equivalents to mt_edge, mt_inpand, mt_expand, mt_inflate and mt_deflate.
https://github.com/chikuzen/tweak

[Myrsloik]

Interesting, I thought I'd made interleave identical. I think my way is more intuitive that you always get (number of clips)*(longest clip) frames back. I think I won't change it unless someone can think of a very good reason. Small avisynth differences isn't such a big issue in general.

I also fixed the documentation so it will appear correct in r18.

[kolak]

Is it possible to have overlay timecode filter (maybe using just implemented subtitles filter), with simple size and position options.
Is timecode implemented as metadata already?
It would be nice if we could make it visible and have ability to overwrite/specify start TC (and have drop-frame TC also).

[Myrsloik]

Is it possible to have overlay timecode filter (maybe using just implemented subtitles filter), with simple size and position options.
Is timecode implemented as metadata already?
It would be nice if we could make it visible and have ability to overwrite/specify start TC (and have drop-frame TC also).

It should be possible to do once lachs0r actually gets around to compiling a windows version of the subtitle filter. Libass is a bit annoying so I'd rather avoid compiling that stuff.

Currently source filters only add frame duration but I plan to add more frame information to both FFMS2 and AVISource in the future. Such as the original absolute timestamp.

[Gavino]

Interesting, I thought I'd made interleave identical. I think my way is more intuitive that you always get (number of clips)*(longest clip) frames back.
Your way also seems more intuitive to me and is what the Avisynth documentation actually implies.
However, the code does it the other way and this seems to be what the original author intended - see discussion here.

[cretindesalpes]

I think I won't change it unless someone can think of a very good reason.

Typical case:

a = SelectEvery (3, 0)
b = SelectEvery (3, 1)
c = SelectEvery (3, 2)
# Some processing on a, b or c, then:
Interleave (a, b, c)

You generally expect Interleave to return the same number of frames as the input clip.

[Myrsloik]

Typical case:

a = SelectEvery (3, 0)
b = SelectEvery (3, 1)
c = SelectEvery (3, 2)
# Some processing on a, b or c, then:
Interleave (a, b, c)

You generally expect Interleave to return the same number of frames as the input clip.

That's a good reason, I'll make the used behavior an option.

[qyot27]

On 32-bit Linux:
$ ./waf build
Waf: Entering directory `/home/qyot27/vapoursynth/build'
[ 2/21] asm: src/core/asm/expr.asm -> build/src/core/asm/expr.asm.1.o
[ 2/21] asm: src/core/asm/transpose.asm -> build/src/core/asm/transpose.asm.1.o
[ 3/21] asm: src/core/asm/x86check.asm -> build/src/core/asm/x86check.asm.1.o
[ 4/21] asm: src/core/asm/x86cpu.asm -> build/src/core/asm/x86cpu.asm.1.o
[ 5/21] qxx: src/core/cachefilter.cpp -> build/src/core/cachefilter.cpp.1.o
[ 6/21] qxx: src/core/cpufeatures.cpp -> build/src/core/cpufeatures.cpp.1.o
[ 7/21] qxx: src/core/exprfilter.cpp -> build/src/core/exprfilter.cpp.1.o
../src/core/asm/expr.asm:323: error: elf: invalid relocation (WRT or size)
Waf: Leaving directory `/home/qyot27/vapoursynth/build'
Build failed
-> task in 'objs' failed (exit status 1):
{task 154408300: asm expr.asm -> expr.asm.1.o}
['/usr/local/bin/yasm', '-w', '-Worphan-labels', '-Wunrecognized-char', '-Dprogram_name=vs', '-DARCH_X86_64=0', '-felf32', '-I/home/qyot27/vapoursynth/build/include', '-I/home/qyot27/vapoursynth/include', '-I/usr/include/qt4', '-I/usr/include/qt4/QtCore', '../src/core/asm/expr.asm', '-o', 'src/core/asm/expr.asm.1.o']

Reverting r398 makes it compile again.

[Myrsloik]

Sigh... left in some junk from experimenting...
Will unbreak it tomorrow. Writing asm that works everywhere certainly is annoying.

[sl1pkn07]

works on arch64 (r404)

http://paste.ubuntu.com/1481082

patch "fixes" a few warnings http://paste.ubuntu.com/1481085

cython 0.17.2
python 3.3.0
yasm 1.2.0
waf 1.7.8
qt 4.8.4
ffmpeg 1.0.1
libass-git 7e4553

if help anything

[jackoneill]

http://paste.ubuntu.com/1481082

patch "fixes" a few warnings http://paste.ubuntu.com/1481085


Regarding the qFatal() calls: why are you eating the spaces?

[Myrsloik]

Regarding the qFatal() calls: why are you eating the spaces?

And why do you hate empty strings in python? I'll apply the qfatal stuff with spaces tomorrow but the rest is rejected. The precedence of logic operators is also well defined so I don't intend to change it just to please a very high compiler warning level.

[jackoneill]

And why do you hate empty strings in python? I'll apply the qfatal stuff with spaces tomorrow but the rest is rejected. The precedence of logic operators is also well defined so I don't intend to change it just to please a very high compiler warning level.

I think both gcc and clang warn about those empty strings, but they do the job anyway.

[Myrsloik]

I'm finally going completely crazy. I have narrowed down the python memory leak quite a bit but I just can't explain the last part. If anyone happens to be good at debugging python (perhaps you've even used cython too) memory leaks and want to help please contact me. I have a fairly simple case that can reproduce the issue but I simply don't know what triggers it in the first place.

Unless this bug is found it will never be possible to simply press F2 again and again in vdub to refresh a script.

[Adub]

quick update for Waf version.

Index: bootstrap.py
===================================================================
--- bootstrap.py (revision 405)
+++ bootstrap.py (working copy)
@@ -2,7 +2,7 @@

import os, stat, urllib2

-f = urllib2.urlopen('http://waf.googlecode.com/files/waf-1.7.6')
+f = urllib2.urlopen('http://waf.googlecode.com/files/waf-1.7.8')

with open('waf', 'wb') as waf:
waf.write(f.read())


BTW, Myrsloik, do you have a preferred method in which we send patches to you?

[Myrsloik]

quick update for Waf version.

Index: bootstrap.py
===================================================================
--- bootstrap.py (revision 405)
+++ bootstrap.py (working copy)
@@ -2,7 +2,7 @@

import os, stat, urllib2

-f = urllib2.urlopen('http://waf.googlecode.com/files/waf-1.7.6')
+f = urllib2.urlopen('http://waf.googlecode.com/files/waf-1.7.8')

with open('waf', 'wb') as waf:
waf.write(f.read())


BTW, Myrsloik, do you have a preferred method in which we send patches to you?

I'm not that picky but if you open an issue for it on the bug tracker and add it as an attachment I definitely won't miss it. I'm happy as long as it's in some kind of diff/patch format, really.

Anyway, applied.

[Adub]

Cleanup for some of the invert_example.c comments: https://gist.github.com/4450489

[Myrsloik]

Cleanup for some of the invert_example.c comments: https://gist.github.com/4450489

Applied. I've also finally managed to solve most of the evils of python so the next version will leak a lot less memory.

[Adub]

Applied. I've also finally managed to solve most of the evils of python so the next version will leak a lot less memory.

Excellent, good to hear. I've begun work on my CUDA environment. I'm getting used to the Waf build system and incorporating the CUDA compiler into the mix. It's almost up an ready for dev, but I've got a few CFLAGS issues to fix first.

[Myrsloik]

Will the missing operator ^(power) in Expr be available in the next version as well? :D

I have no optimized code for it so just use exp and log to get the same result.

[Reel.Deel]

I have no optimized code for it so use exp and log to get the same result.

For a RPN/LUT dummy like myself, how would I write this mt_lut expression using Expr?
mt_lut("x 128 - 1.49 * x 128 - 2 ^ x 128 - 2 ^ 9 + / * 128 +")

I also think it would be a good idea for existing mt_lut/x/y/z expressions to be compatible with Expr with no additional gimmicks.

[Myrsloik]

For a RPN/LUT dummy like myself, how would I write this mt_lut expression using Expr?
mt_lut("x 128 - 1.49 * x 128 - 2 ^ x 128 - 2 ^ 9 + / * 128 +")

I also think it would be a good idea for existing mt_lut/x/y/z expressions to be compatible with Expr with no additional gimmicks.

Well... theoretically I could add the ^ operator but it would do you no good since it's very slow just for squaring things. In this case x*x is much faster. So I give you this equivalent that doesn't even need ^ and removes some duplicate evaluation:
"x 128 - 1.49 * x 128 - dup * dup 9 + / * 128 +"
(warning, completely untested but should work and if it doesn't just tell me and I'll actually try it myself)

When you go from a lut to actually evaluating the expression for every pixel you have to be more careful about how you write it too or it could become slow.

Bonus material by jfs, the original expression in readable form:
(x-128)*1.49 * (((x-128)^2) / ((x-128)^2 + 9)) + 128