Vapoursynth

[Myrsloik]

Transpose crashes python.exe
vs-v8(redownload)
python-3.2.3

That's not much to go on. It crashes with different input formats? Does your cpu have sse2?

[Are_]

Yeah, sorry for my poor report, after further testing:

Input is yv12 coming from native ffms2 (BD m2ts)
As long as the total sum of pixels is less than 2000000, it does not seem to crash (only played a little resizing before Transpose)
Tested on my amd cpu and an i3 friends's one, both with sse2

[Myrsloik]

Yeah, sorry for my poor report, after further testing:

Input is yv12 coming from native ffms2 (BD m2ts)
As long as the total sum of pixels is less than 2000000, it does not seem to crash (only played a little resizing before Transpose)
Tested on my amd cpu and an i3 friends's one, both with sse2

Will be fixed in the next release.

[Myrsloik]

Seems "global" is a special statement to Python, how could I get around this? TIA.


>>> super = core.avs.MSuper(clip, mt=False)
>>> bvec2 = core.avs.MAnalyse(super, isb=True, delta=2, blksize=16, overlap=8, mt=False, truemotion=False, global=True)
File "<stdin>", line 1
bvec2 = core.avs.MAnalyse(super, isb=True, delta=2, blksize=16, overlap=8, mt=False, truemotion=False, global=True)
^
SyntaxError: invalid syntax
>>>


You can put all the arguments in a dict and then pass the dict as function(**dict)
I'll think about what to do with it in the next version. Maybe some kind of automatic argument renaming would be a good idea...

[-Vit-]

Finally had time to try some scripting, ultimate aim to get (Q)TGMC working. Hit some problems:

- No Bob, Merge or TemporalSmooth
-- I cannot replace Bob with the SeparateFields/Resize equivalent because the resizers don't support float sample offsets as they do in Avisynth
-- Merge can be replaced with masktools calls, but will be slower
-- TemporalSmooth can be kinda replaced with trim and masktools work, but will be much slower and really needs scene change awareness. Any suggestions for an existing external plugin that does the same (MVTools can do it with null vectors, but that seems overkill), or does it need a new plugin to be written?

- I have a problem with Interleave (unless I'm being stupid). This script should do nothing, but actually returns a clip with (almost?) every other frame dropped:

#...
e = core.std.SelectEvery(clip=c,cycle=2,offsets=[0]) # SelectEven
o = core.std.SelectEvery(clip=c,cycle=2,offsets=[1]) # SelectOdd
c = core.std.Interleave(clips=[e,o])
#...


- Changelog says unnamed parameters are now accepted, how?

[hajj_3]

python 3.3 will be out within 1 week btw, RC3 is currently out so you can switch to VS2010 once that is released :)

[Myrsloik]

Finally had time to try some scripting, ultimate aim to get (Q)TGMC working. Hit some problems:

- No Bob, Merge or TemporalSmooth
-- I cannot replace Bob with the SeparateFields/Resize equivalent because the resizers don't support float sample offsets as they do in Avisynth
-- Merge can be replaced with masktools calls, but will be slower
-- TemporalSmooth can be kinda replaced with trim and masktools work, but will be much slower and really needs scene change awareness. Any suggestions for an existing external plugin that does the same (MVTools can do it with null vectors, but that seems overkill), or does it need a new plugin to be written?

- I have a problem with Interleave (unless I'm being stupid). This script should do nothing, but actually returns a clip with (almost?) every other frame dropped:


- Changelog says unnamed parameters are now accepted, how?


e = core.std.SelectEvery(c,2,[0]) <- that's all there is to it, simple


It's well known that the resizers suck. I'm looking into several different options to fix it.

You're right. No bob. Maybe you can use some other bob filter and tweak the settings to make it really stupid.

I plan to start work on a kind of masktools replacement based on asmjit after R9 is released, it should be a good base for making the merge functionality you want.

Temporalsmooth is also just another temporal smoother. I don't think it belongs as a base filter.

Maybe someone will come along and rip out some of avisynth's more popular filters and make them standalone plugins. I think it's the best interim solution.

I found the interleave bug. A complete brainfart at its best when it was written.

[-Vit-]

e = core.std.SelectEvery(c,2,[0]) <- that's all there is to it, simple

I tried that and get this:
TypeError: __call__() takes exactly 0 positional arguments (3 given)

[Myrsloik]

You most likely have an old version lying around without realizing it. Delete all copies of vapoursynth.pyd and vscore.dll and reinstall.

[-Vit-]

So I did. I should know better...

[Mug Funky]

here's a replacement for bob, at least using avisynth code (i haven't had time to teach myself python yet):

interleave(separatefields().selecteven().bicubicresize(last.width,last.height,1/3.,1/3.,0,.25,last.width,last.height/2),separatefields().selectodd().bicubicresize(last.width,last.height,1/3.,1/3.,0,-.25,last.width,last.height/2))

i think vapoursynth should have enough internal filters to implement the above line, if the resizers are up to it.

[AzraelNewtype]

here's a replacement for bob, at least using avisynth code (i haven't had time to teach myself python yet):

interleave(separatefields().selecteven().bicubicresize(last.width,last.height,1/3.,1/3.,0,.25,last.width,last.height/2),separatefields().selectodd().bicubicresize(last.width,last.height,1/3.,1/3.,0,-.25,last.width,last.height/2))

i think vapoursynth should have enough internal filters to implement the above line, if the resizers are up to it.

The resizers only handle integer offset values, it's literally the problem -Vit- said he had with trying to implement it by hand himself.

[Myrsloik]

I've released R9. See http://www.vapoursynth.com/2012/09/r9-the-mostly-done-release/ for the full post with changes and what's on my todo list.

Here's a small example of the new slicing possible with clips in Python:
clip[1:] # equivalent to Trim(c, first=1)
clip[:9] # equivalent to Trim(c, last=8)
clip[1::2] #equivalent to selecting all frames with odd numbers
clip[::-1] # Reverse(c)
Interleave([clip[::2], clip[1::2]]) # do "nothing"
clip1[200::-5] + clip2[5:10:3] # ponder this one

[kolak]

Can't wait to see vfw module, even if I'm still not sure how useful it will be :)

[Myrsloik]

The slicing result seems to be wrong or do i do anything wrong :confused:

You found the single case where I forgot to adjust the trim end appropriately (with only stop and not start set). I've reuploaded R9 with that small detail fixed.

Note that clip[0:9] will give you the correct result even in the broken version.

[-Vit-]

Trying to do some temporal scripting. Couple of queries:
- SelectEvery fails if any selected frame is negative (doesn't seem to be so much a problem at the clip end...?). Avisynth duplicates the start/end frames, so allowing "out-of-cycle" offsets. This is used by many avisynth scripts. Without it, temporal processing at the clip ends becomes trickier.
- Related, SelectEvery does not accept a cycle of 1. For selecting the clip offset by a number of frames: SelectEvery(c,1,[-2])

- Negative slice start and end values? I thought they counted from the end of the sequence. Seems to reject them here. Exacerbates the above two points because the workarounds involve padding at the start and the end.

- Suggestion: * operator for clip repetition. c * 2 = clip repeated twice.

[Myrsloik]

Trying to do some temporal scripting. Couple of queries:
- SelectEvery fails if any selected frame is negative (doesn't seem to be so much a problem at the clip end...?). Avisynth duplicates the start/end frames, so allowing "out-of-cycle" offsets. This is used by many avisynth scripts. Without it, temporal processing at the clip ends becomes trickier.
- Related, SelectEvery does not accept a cycle of 1. For selecting the clip offset by a number of frames: SelectEvery(c,1,[-2])

- Negative slice start and end values? I thought they counted from the end of the sequence. Seems to reject them here. Exacerbates the above two points because the workarounds involve padding at the start and the end.

- Suggestion: * operator for clip repetition. c * 2 = clip repeated twice.

Trim splice and repeat is for offsetting a clip. Why you need selectevery to do so is a bit of a mystery to me. Seems like you're stuck in your avisynth ways. I suppose I could allow out of cycle offsets in selectevery if you have a good use case.
Also, negative offsets in slicing is just crazy. It goes against all that is good and python. Use clip[clip.num_frames - 10, clip.num_frames - 5, -1] If you need something similar. I intentionally kept trim simple. Look at the mess in avisynth and you'll see why.
I'll think about a repeat operator but it's a lot less common for someone to need it. I want to keep it simple and avoid operator overloading hell.

[-Vit-]

I'm was trying to do a 2-radius temporal smooth in script for TGMC. The trick is the clip ends, where there aren't enough frames. Naturally I started with the avisynth approach because it deals with that trivially; and there are no complex vapoursynth script examples - you gotta start somewhere.

Also, negative offsets in slicing is just crazy. It goes against all that is good and python.
I rarely code Python so I don't have that Pythony mindset... but negative indices are allowed in ordinary Python list slicing so why are they not good Python?

I'll think about a repeat operator but it's a lot less common for someone to need it. I want to keep it simple and avoid operator overloading hell.
It's useful for the padding needed at the ends of the clip I needed to implement the above. I guess it can be done with slice+loop. But you go against expectations by providing some list operators and not others (I immediately went for the * operator and I'm not a Pythoner)

[Myrsloik]

I'm was trying to do a 2-radius temporal smooth in script for TGMC. The trick is the clip ends, where there aren't enough frames. Naturally I started with the avisynth approach because it deals with that trivially; and there are no complex vapoursynth script examples - you gotta start somewhere.


I rarely code Python so I don't have that Pythony mindset... but negative indices are allowed in ordinary Python list slicing so why are they not good Python?


It's useful for the padding needed at the ends of the clip I needed to implement the above. I guess it can be done with slice+loop. But you go against expectations by providing some list operators and not others (I immediately went for the * operator and I'm not a Pythoner)

Right, I missed the part with negative indexes. It will be added. I try to be pythonish whenever possible. I suppose I'll add * too just to see how it turns out.

As for scripts I suggest you post a few lines of avisynth script here and I'll tell you how it'd write it. I'm actually surprised so few people ask me about the syntax.

[lansing]

ok i'lve read all 12 pages of this thread, and i have the script ready in a py file. But how do you run it with different codec like xvid, divx and x264? I mean how do I write it in the command window? And how do I measure the speed of each filters without outputing the video file?

And there's no link in the documentation page that redirect back to the homepage

[Myrsloik]

ok i'lve read all 12 pages of this thread, and i have the script ready in a py file. But how do you run it with different codec like xvid, divx and x264? I mean how do I write it in the command window? And how do I measure the speed of each filters without outputing the video file?

And there's no link in the documentation page that redirect back to the homepage

1. See the first post on how to pipe to x264.exe.
2. Pipe to NUL (script.py > NUL) to not write a file.
3. I usually use time in mingw when just doing quick timing.

[Reel.Deel]

Thanks for the new VapourSynth documentation, it's compact yet very informative.

So I noticed that when using AVS plugins in VS, boolean parameters are now integer. Was this how you intended it to be or is it because of the Python interpreter?

[Myrsloik]

Thanks for the new VapourSynth documentation, it's compact yet very informative.

So I noticed that when using AVS plugins in VS, boolean parameters are now integer. Was this how you intended it to be or is it because of the Python interpreter?

It's a design decision in vs to keep the number of types down so bool was left out and frames and functions added instead.
I've also marked all arguments that work like a bool with bint in the documentation if anyone's wondering what it means.
It has absolutely nothing to do with python.

[Chikuzen]

@Myrsloik
Although it seems that you stop using B tag with Y4M output and added like 420p16 to C tag, I think that this is not good.
C tag also has the role which specifies chroma position other than a colorspace specification.
You should not give the function more than now to C tag.

If you want to process over 8bit format with Y4m, you should use new B tag.
And it will be better to decide that W becomes half if value of B is 9 or more (like dither-hack's interleaved format).
At least, compatibility is not be broken if it carries out like this.
A Y4M purser which does not refer to B tag will only process it as a collapsed image with twice width.

[Myrsloik]

@Myrsloik
Although it seems that you stop using B tag with Y4M output and added like 420p16 to C tag, I think that this is not good.
C tag also has the role which specifies chroma position other than a colorspace specification.
You should not give the function more than now to C tag.

If you want to process over 8bit format with Y4m, you should use new B tag.
And it will be better to decide that W becomes half if value of B is 9 or more (like dither-hack's interleaved format).
At least, compatibility is not be broken if it carries out like this.
A Y4M purser which does not refer to B tag will only process it as a collapsed image with twice width.


Actually that change is to be more compatible. It's how ffmpeg already implemented it so you can pipe directly to it now.

[lansing]

i'm having problem working with ts files using ffms2. It's getting really slow when parsing in a large ts file, like 30G, where indexing took like 15 minutes and encoding speed went down to 2fps. I tested to trim a small portion (4000 frame) of the same ts file, indexing was like instant and speed constant at 40fps. Don't know what is causing the let down.

And I'm getting rare framerate on the output video, like 4382/143 fps, where it supposed to be 25fps.

[Chikuzen]

Actually that change is to be more compatible. It's how ffmpeg already implemented it so you can pipe directly to it now.

oh, I hope libav does not take this in.

[Myrsloik]

i'm having problem working with ts files using ffms2. It's getting really slow when parsing in a large ts file, like 30G, where indexing took like 15 minutes and encoding speed went down to 2fps. I tested to trim a small portion (4000 frame) of the same ts file, indexing was like instant and speed constant at 40fps. Don't know what is causing the let down.

And I'm getting rare framerate on the output video, like 4382/143 fps, where it supposed to be 25fps.

This is a known ffms2 issue. It has never been good with transport streams.

[Myrsloik]

oh, I hope libav does not take this in.

I intend to port it soon. Actually it's already a mess because y4m only has different color positions defined for yuv420p8. Anything else and it doesn't. I'd say this is the proper way would be to add flags for colorimetry, range and chroma position in new fields. But y4m is already a mess and nobody would support it properly anyway...

[gnaggnoyil]

It seems that TFM and TDecimate can't work normally and properly under vapoursynth. Is there any replaceable filters to do the ITVC?
Since the lgd files I use in delogo filters must work after the ITVC is complete, it would be very helpful for me to solve the problem.
Or perhaps I had to do it in mencoder? lol.

--------------------

I use TDM and TDecimate like:

video=core.avs.TFM(c1=video,order=1)
video=core.avs.TDecimate(c1=video,mode=1,hybrid=1)

and in the command prompt the script warns like this:

Avisynth Compat: requested frame XXX not prefetched, using slow method that may deadlock

[Myrsloik]

I use TDM and TDecimate like:

video=core.avs.TFM(c1=video,order=1)
video=core.avs.TDecimate(c1=video,mode=1,hybrid=1)

and in the command prompt the script warns like this:

Avisynth Compat: requested frame XXX not prefetched, using slow method that may deadlock

It's a warning, not an error. Try reading next time.

[gnaggnoyil]

It's a warning, not an error. Try reading next time.

:scared:But it seems there is something wrong which is slowing the speed of the script, any method to solve it?

[Myrsloik]

:scared:But it seems there is something wrong which is slowing the speed of the script, any method to solve it?

Start porting TDecimate. That's the solution. The filter has really odd frame request patterns and because of that it's impossible to prefetch all the input frames it needs.
You can also try your luck with TFM+Decimate from decomb. It may actually end up a bit faster in vs.
But yeah... tritical makes filters with good ideas but very messy code. Porting these things will take time.

[Chikuzen]

:scared:But it seems there is something wrong which is slowing the speed of the script, any method to solve it?

#script_1.py
import vapoursynth as vs
import sys
core = vs.Core(threads=4)
core.avs.LoadPlugin("TIVTC.dll")
core.avs.LoadPlugin("DGDecode.dll")
video = core.avs.MPEG2Source("foo.d2v", idct=3)
video = core.avs.TFM(video, order=1)
video = core.avs.TDecimate(video, mode=1,hybrid=1)
video.output(sys.stdout, y4m=False)


$time python3 script_1.py > /dev/null 2>&1
real 2m1.805s
user 0m0.000s
sys 0m0.015s


#script_2.py
import vapoursynth as vs
import sys
core = vs.Core(threads=4)
core.std.LoadPlugin("vsavsreader.dll")
lines = '''
LoadPlugin("TIVTC.dll")
LoadPlugin("DGDecode.dll")
MPEG2Source("foo.d2v", idct=3)
TFM(order=1)
TDecimate(mode=1,hybrid=1)
'''
video = core.avsr.Eval(lines)
video.output(sys.stdout, y4m=False)


$ time python3 script_2.py > /dev/null
real 1m44.076s
user 0m0.000s
sys 0m0.045s


you can get speed comparable as avisynth when using vsavsreader.

[Myrsloik]

I've released R10. Enjoy the vfw stuff. Just remember that vs does not do error message clips, instead opening the file fails. Only applications like vdub that query the error information interface and the python command line will show the actual error message.

Also, please consider donating to the effort of making a complete cross platform frameserver.

Some more information and details in my blog post here:
R10 – VFW enters the scene from the left (http://www.vapoursynth.com/2012/09/r10-vfw-enters-the-scene-from-the-left/)

[lansing]

this is just amazing to see how fast vapoursynth grow in just a month, I foresaw the dimming of avisynth within the next two months at this pace. Hats to you Myrsloik

[lansing]

I just updated to r10, but can't seems to get the preview working.

I renamed the py file to vpy, followed the instruction to open in virtualdub, but got a out of bound memory crash after opening it. And MPC-HC status just showing "opening..."
Am I doing something wrong?

[Myrsloik]

I just updated to r10, but can't seems to get the preview working.

I renamed the py file to vpy, followed the instruction to open in virtualdub, but got a out of bound memory crash after opening it. And MPC-HC status just showing "opening..."
Am I doing something wrong?

I forgot to mention the most important part :D
Scripts you want open in this way need to have the returned clip assigned to the variable last and cannot use clip.output().
An example:
ret = source filter here
ret = other filters(ret)
last = ret
Then try to open it in vdub.

[kolak]

Vfw came so fast, but what about software, which does open files by extention? Even if it works with vfw it will fail as extension won't be avi. Making script seen by other apps as "fake" avi (or mov) would be great. This would make it 99% compatible with all apps.
Are 8bit+ color spaces/outputs coming? Again v210 would be great as this is most compatible 10bit format.

I should make a use of this donatation button :)

[lansing]

stilll no luck, still crashed :(


import vapoursynth as vs
import sys
core = vs.Core(accept_lowercase=True)

# plugins
core.avs.LoadPlugin(r"C:\Program Files (x86)\AviSynth 2.5\plugins\DGDecode.dll")

# load source
src = core.avs.MPEG2Source(d2v=r"test.d2v")

last=src

[Myrsloik]

Vfw came so fast, but what about software, which does open files by extention? Even if it works with vfw it will fail as extension won't be avi. Making script seen by other apps as "fake" avi (or mov) would be great. This would make it 99% compatible with all apps.
Are 8bit+ color spaces/outputs coming? Again v210 would be great as this is most compatible 10bit format.

I should make a use of this donatation button :)

For the fake avi stuff you need another layer like that old vfapi stuff or avfs. I could try to quickly port avfs and make it vsfs if the current solution isn't good enough.

I looked at v210 but it's packing made me nauseous just to look at. I suppose I'll add it sooner or later but it's a shame it turned out to be the most popular. Microsoft defines very sane and clean planar formats officially too. What a shame... over 8 bit output will come. But it's a bit lower in priority since to my knowledge x264 is the only thing that truly benefits from it. Correct me if I'm wrong.

[Myrsloik]

stilll no luck, still crashed :(


import vapoursynth as vs
import sys
core = vs.Core(accept_lowercase=True)

# plugins
core.avs.LoadPlugin(r"C:\Program Files (x86)\AviSynth 2.5\plugins\DGDecode.dll")

# load source
src = core.avs.MPEG2Source(d2v=r"test.d2v")

last=src



Then I'm not sure why it fails. Try running the program from the command-line. Maybe you'll see a warning or error printed before it crashes. You can also try blankclip and see if that works.

[Reel.Deel]

VS VFW works fine for me. Thanks Myrsloik.

I'm assuming it will be up to VDub to support drag and drop for vpy or am I wrong?

[Myrsloik]

VS VFW works fine for me. Thanks Myrsloik.

I'm assuming it will be up to VDub to support drag and drop for vpy or am I wrong?

Correct. Vdub needs to be updated to know about the .vpy extension. Feel free to go and push for its inclusion in the next vdub version. Along with support for the sane high bitdepth planar formats.

[Reel.Deel]

Feel free to go and push for its inclusion in the next vdub version. Along with support for the sane high bitdepth planar formats.

Will do. :D

[lansing]

Then I'm not sure why it fails. Try running the program from the command-line. Maybe you'll see a warning or error printed before it crashes. You can also try blankclip and see if that works.

can you elaborate how to run it the command line way?

[Myrsloik]

can you elaborate how to run it the command line way?

Start menu\open command prompt
Cd virtualdub path
Virtualdub

Done.

Another thing you should try first though is only having absolute paths in your script. The other thing is running your script in python and see if it works at all with the paths you have.

[Keiyakusha]

Works fine so far thank you so much! Now I can do some real stuff with it!
But I have somewhat unrelated question.
Should I load ffms2 as std or avs plugin? Using latest ffms2 build it is loaded via std yet says it "doesn't have function FFVideoSource". I think I missed something. How should I call it then?

[Chikuzen]

Works fine so far thank you so much! Now I can do some real stuff with it!
But I have somewhat unrelated question.
Should I load ffms2 as std or avs plugin? Using latest ffms2 build it is loaded via std yet says it "doesn't have function FFVideoSource". I think I missed something. How should I call it them?

http://forum.doom9.org/showpost.php?p=1590957&postcount=1650

[active1]

Correct. Vdub needs to be updated to know about the .vpy extension. Feel free to go and push for its inclusion in the next vdub version. Along with support for the sane high bitdepth planar formats.

maybe someone should add the .vpy extension support in x264 too (is it possible to do it?)
EDIT: sorry, it works :p
EDIT: lol, no it's not working, sorry :stupid:
that's what happend:
avs [info]: trying FFmpegSource2... indexing... failed
avs [info]: trying DSS2... not found
avs [info]: trying DirectShowSource... succeeded
avs [info]: 848x480p 0:0 @ 10000000/417013 fps (cfr)

the x264 loaded the test.vpy script through directshowsource