avs2pipe [Archive] - Page 2

Gavino

5th April 2011, 11:32

would you have any issues with me adding auto loading of scripts via GImport? I can see two easy ways, a) Just load all scripts via GImport if the function / dll is present, or b) trying to load it if the passed file is *.gscript (or similar). Do you have a preference?

I.E. You would be able to do avs2pipe video myscript.gscript and have it autoload the plugin and GImport the script...
Sounds like a cool idea either way. I don't have a strong preference, but if going for option b, how about .avsg as the extension (based on existing use of .avsi)?

doobry

5th April 2011, 11:38

using an avsg extension would be my preference, as it means that scripts with GScript features would be harder to mistake for "normal" avs scripts, either linked on forums or "found" years after they were written.

doobry

5th April 2011, 13:01

avs2pipe [info]: writing 20 seconds of 0 Hz, 48000 channel audio.

I went through and changed a bunch of the standard fprintf placeholders to the super special Microsoft versions in the last release (0.0.3), are you still having this problem as I cannot reproduce it?

If its still there what flavor (vs or gcc etc) and type of input are you using?

tebasuna51

5th April 2011, 14:56

I went through and changed a bunch of the standard fprintf placeholders to the super special Microsoft versions in the last release (0.0.3), are you still having this problem as I cannot reproduce it?

Is ok now, thanks.

IanB

6th April 2011, 00:20

@doobry,

In terms of plane order, generally you cannot assume a particular order in memory. In all places except env->NewVideoFrame(vi), CS_YV12 and CS_I420 are considered identical. This distinction is to allow source filters to import YV12 and I420 video data directly into a PVideoFrame without needing to blit the individual planes.

doobry

6th April 2011, 12:31

In terms of plane order, generally you cannot assume a particular order in memory. In all places except env->NewVideoFrame(vi), CS_YV12 and CS_I420 are considered identical. This distinction is to allow source filters to import YV12 and I420 video data directly into a PVideoFrame without needing to blit the individual planes.

Given this does reading via avs_get_read_ptr_p(frame, planes[p]) etc, always return the correct pointer to Y, U, V plane when using the AVS_PLANAR_Y, AVS_PLANAR_U, AVS_PLANAR_V constants? Assuming that another filter has not messed something up?

i.e. Is avs2pipe doing it right? The only testing I can do easily is using FFMpegSource which marks the stream I420, and DGIndex which marks the same source as YV12... Both given identical output with avs2pipe, using the same YUV4MPEG2 generation method, so it works for me, but I just want to make sure its not a fluke.

Gavino

6th April 2011, 13:18

Given this does reading via avs_get_read_ptr_p(frame, planes[p]) etc, always return the correct pointer to Y, U, V plane when using the AVS_PLANAR_Y, AVS_PLANAR_U, AVS_PLANAR_V constants? Assuming that another filter has not messed something up?
Yes, because GetReadPtr (or avs_get_read_ptr_p in the C API) uses the appropriate offset (offsetU, offsetV) from the frame header (previously stored by NewVideoFrame(vi)).

IanB

7th April 2011, 00:37

Given this does reading via avs_get_read_ptr_p(frame, planes[p]) etc, always return the correct pointer to Y, U, V plane when using the AVS_PLANAR_Y, AVS_PLANAR_U, AVS_PLANAR_V constants? Assuming that another filter has not messed something up?

i.e. Is avs2pipe doing it right? The only testing I can do easily is using FFMpegSource which marks the stream I420, and DGIndex which marks the same source as YV12... Both given identical output with avs2pipe, using the same YUV4MPEG2 generation method, so it works for me, but I just want to make sure its not a fluke.
For testing SwapUV() with a planar just flips the pointers in the header (zero cost), so you can make sure you code is plane order agnostic.

doobry

7th April 2011, 12:20

Thanks to IanB and Gavino causing me to look though the color space handling code, I've rewritten it for the "next" version. It now uses an internal buffer to organize each frame, before writing it to stdout with a single call to fwrite. Before it called fprintf for the FRAME header, and then fwrite for each row of data...

This has given me a huge speed boost, ~1.3 seconds to write 1000 frames to null, vs ~2.1 seconds using the old method, however I noticed that it will no longer output to the windows console, not a problem as you should never want to do that, but it did make me wonder if there is a limit to how much you can write to a pipe with a single fwrite... it still works fine via x264 or piping to a file, but I want to be sure.

So before I upload this and remove the link to the working version, does anyone want to have a test, or have any views on this... I will provide a download link via PM to a test version, but I'm not posting it here as I've noticed scrapers are linking to my downloads, which I don't really mind but I don't want some random site posting a possibly broken version without the info saying it might be busted.

Obviously this "huge" speed increase will be pretty insignificant vs the slowdown caused by say piping the output to x264 but it all helps.

Chikuzen

7th April 2011, 16:27

doobry

7th April 2011, 16:40

@doobry
It seems that x264 team schedules to change some encode options.
https://github.com/DarkShikari/x264-devel/commit/4a5f4432172b33bfaef1e2f6c63c468060efe1e4

I think that you might discontinue "x264bd" because it will be outdated soon.

From that commit link...

This option doesn't automatically enforce every aspect of Blu-ray compatibility (e.g. resolution, framerate, level, etc).

As I said when I added it, its really a feature for me, so that I don't have to refer to the website and lookup the correct options for the file i'm encoding... It does however error out when it cannot find a matching "profile" for the video, something that the commit comment says the changes do not do.

It would be great if the same feature ended up in x264 as it would mean I can be 100% sure i'm doing the right thing, x264bd is just a consolidation of info on the web, for my use. Bring on the day I can remove it can have a --profile bluray option on x264 that checks everything (framerate, resolution, interlacing etc) will be correct.

And if someone is relying on it, then I'll just modify the options to match the new guide... when someone writes one :D

doobry

7th April 2011, 21:24

Did a little benchmarking, real time in seconds, run 3 times each.

avs2pipe 0.0.3:

avs2pipe26_vs 54.23 54.06 54.01
avs2pipe26_gcc 51.83 52.60 52.10
avs2pipe_vs 54.35 54.27 54.26
avs2pipe_gcc 51.86 52.10 51.97

avs2pipe 0.0.4 alpha:

avs2pipe26_vs 32.99 32.73 32.84
avs2pipe26_gcc 32.85 32.72 32.82
avs2pipe_vs 32.81 32.84 32.95
avs2pipe_gcc 32.80 32.73 32.76

avs2yuv 0.24:

downloaded 45.91 46.43 46.11
vs2010 express 54.20 54.44 54.00

Test Was:

Windows 7 x64, Quad Core AMD, AviSynth 2.6 Alpha 2
32374 frames 720x576 YUV420 video from PAL DVD
FFMpegSource for m2v file
FFMpegSource for ac3 file (Not Processed)
Output piped to NUL
MSYS bash time used to time execution
MSYS md5sum used once to check output identical

Thing that really surprised me was that opening the avs2yuv project in Visual Studio 2010 Express and compiling it reduced it speed from around 46sec to 54sec... Is the Express compiler really that bad?

Chikuzen

7th May 2011, 23:24

Hi.
I fiddled with the code of this tool for about one month.
(The purpose is to make it to more wieldy thing for me, and learning C language.)

binary
deleted

source code
https://github.com/chikuzen/avs2pipemod

Differences with original
* Display total elapsed time.
* contents of 'info'
* New option 'benchmark'.
* 'y4mp', 'y4mt', 'y4mb' and 'rawvideo' options instead of 'video'.
* Add SAR info to yuv4mpeg2 stream header.
* FieldBased input will be corrected to framebased on yuv4mpeg2 output mode.
* Colorspace conversion that takes colormatrix and interlace into consideration.
* 'rawaudio' option.
* Convert bit depth function in audio output mode.
* Only the compilation with mingw/msys is supported.

b66pak

8th May 2011, 19:24

thanks a lot...
_

Chikuzen

9th May 2011, 15:31

updated.

avs2pipemod-20110510.zip (http://www.mediafire.com/download.php?pl0lx57i7dqsin7)

*Fix incorrect output of 'x264bd', and some modified.

EDIT
Sorry, I found a bug in '-audio' and '-rawaudio'.
Please Download again if you are using avs2pipemod-20110509.zip

Midzuki

9th May 2011, 17:24

@ Chikuzen:

Many many thanks :thanks: :thanks: :thanks:

BTW, ¿ where is doobry ? :)

zambelli

19th May 2011, 22:29

Thanks for this tool, Doobry and Chikuzen. Exactly what I was looking for, and perfect timing too. :)

Chikuzen

3rd July 2011, 07:23

update

avs2pipemod-20110703.zip (http://www.mediafire.com/download.php?aa43daakid35kaa)

add new options '-wav' and '-extwav', and remove '-audio'.

'-wav' outputs audio with normal riff-wave header.
'-extwav' outputs audio with wave extensible header containing channel-mask.

Midzuki

3rd July 2011, 07:54

:thanks: :thanks: :thanks: :thanks: :thanks:

IanB

3rd July 2011, 08:17

add new options '-wav' and '-wavext', and remove '-audio'.
Disappointed you don't honour legacy options, now everybody with -audio has to edit their junk to -wav. :(

But still a great project.

Chikuzen

3rd July 2011, 10:38

Disappointed you don't honour legacy options, now everybody with -audio has to edit their junk to -wav. :(.

no, I changed -audio to -extwav :p

However, I changed it again because what you said was justifiable.
avs2pipemod-20110703-2.zip (http://www.mediafire.com/download.php?9x0du58bii28t27)

@@ -720,6 +720,7 @@ void parse_opts(int argc, char **argv, params *params)
struct option long_opts[] = {
{"wav", optional_argument, NULL, 'w'},
{"extwav", optional_argument, NULL, 'e'},
+ {"audio", optional_argument, NULL, 'e'},
{"rawaudio", optional_argument, NULL, 'a'},
{"y4mp", optional_argument, NULL, 'p'},
{"y4mt", optional_argument, NULL, 't'},

IanB

3rd July 2011, 22:20

Thanks! :D

henryho_hk

2nd August 2011, 01:10

Chikuzen

2nd August 2011, 05:08

CarlEdman

2nd August 2011, 13:33

Is this build multithread-enabled? I have a AMD 1090T X6 and tried:

avs2pip26mod.exe -y4mp some-720p-with-multithread.avs | x264.exe .... --output video.h264 --stdin y4m -
x264.exe .... --output video.h264 some-720p-with-multithread.avs

The first command can only keep one core busy (~16-17%) and the second one can make it ~99%.

Avs2pipe(26)?(mod)? does not need to be MT enabled--all of that is determined by the avisynth library, invoked by both avs2pipe and x264. And the scripts need a "Distributor()" at the end to use multi-threading (a long time ago that was not the case for x264, but it was necessary for quite a while--I have not checked recently if that requirement has gone away again).

So why is one execution multithreaded and the other not? One possibility is that you are using 64-bit x264. Avs2pipe is 32-bit. So if you have, a non-multithreaded 32-bit version 2.6 avisynth.dll installed, but a 64-bit multithreaded avisynth.dll (that is quite possible--the 32-bit and 64-bit system look for the same dll file, but in different paths), then you'd see just what you observe.

Gavino

2nd August 2011, 14:16

So why is one execution multithreaded and the other not? One possibility is that you are using 64-bit x264. Avs2pipe is 32-bit. So if you have, a non-multithreaded 32-bit version 2.6 avisynth.dll installed, but a 64-bit multithreaded avisynth.dll (that is quite possible--the 32-bit and 64-bit system look for the same dll file, but in different paths), then you'd see just what you observe.
The reason is simply that x264 is 'MT-aware' as far as Avisynth is concerned - if it detects that you are running a MT version of Avisynth (and have used SetMTMode), it will make the necessary call to Distributor() itself.

avs2pipe, on the other hand, is not 'MT-aware' so does not call Distributor(), resulting in single threaded running.
(which is why you have to add it yourself.)

Note that for applications using Avisynth via the VfW interface (eg VirtualDub), this is not an issue since in MT Avisynth this path automatically invokes Distributor().

Chikuzen

18th September 2011, 07:14

Why do you want 64bit version ?
There are always avs2yuv and wavi for x64.

Chikuzen

18th September 2011, 07:20

then, use wavi.
http://forum.doom9.org/showpost.php?p=1527102&postcount=4

Chikuzen

18th September 2011, 08:24

I don't understand how to use it. avs2pipemod works fine for me:
avs2pipemod.exe -rawvideo %avs% | C:\Soft\Video\bin\x264-10bit.exe
avs2pipemod.exe -extwav=16bit %avs% > %wav%

If you don't need channel-mask, then
wavi input.avs output.wav
If you need channel-mask, then
wavi input.avs output.wav /x
If you want to output rawvideo, then
avs2yuv input.avs -raw - | x264 - --demuxer raw ...

FYI, currently, it requires avisynth.lib to compile avs2pipe(mod).
However, there is no avisynth.lib for x64.
Thus, it is necessary to rewrite many codes.

Chikuzen

18th September 2011, 09:00

wavi is a program that use vfw interface like VirtualDub/avs2avi/etc...
If wavi cannot treat avs as avi, you will have failed in installation of avisynth.

Chikuzen

19th September 2011, 03:19

update.

avs2pipemod-20110919.zip (http://www.mediafire.com/download.php?p2iyweklw0tp76a)

*add new option 'trim'

SamKook

20th December 2011, 17:45

doobry

29th December 2011, 11:24

Do you still plan on releasing v0.0.4? I'd love to get a similar bump in speed I see with your alpha version benchmark since using this is the only way I found to prevent crashing with memory heavy filters and weird things happen when I use avs2yuv. Also, having everything all in one is pretty great.

The source code is here, there is no tagged 0.0.4 release or binaries, but building it isn't hard

https://github.com/doobry/avs2pipe

Chikuzen

30th December 2011, 19:42

oh doobry! welcome back!

CarlEdman

30th December 2011, 20:03

Ditto to the Welcome Back, doobry.

But could you please make a tagged release with binaries of 0.04? Building binaries is something of a hassle for those without a Windows dev environment.

Chikuzen

30th December 2011, 20:32

update

avs2pipemod-20111230.zip
(http://www.mediafire.com/download.php?uw2z0pzax1dqnf8)

*port video output processing from latest avs2pipe.

b66pak

30th December 2011, 20:57

thanks a lot...
_

CarlEdman

30th December 2011, 23:59

:thanks:

SamKook

31st December 2011, 16:09

Thanks doobry.

It's indeed very easy to compile; change the path to MinGW in the .bat, run it... and you're done.
It compiled so fast that I ran it twice since I was sure it had failed the first time.

And I got a 3.12% speedup in my first test so that's a good thing.

Edit: The second test when encoding them back to back in similar condition is less good though:
h264_10bit_a2p_0-0-3: 0:17:44.77
h264_10bit_a2p_0-0-4: 0:17:42.56
h264_10bit_a2pmod_0-0-3: 0:17:44.16
h264_10bit_a2pmod_0-0-4: 0:17:38.85

Yellow_

2nd January 2012, 22:27

hi

I'm struggling to migrate from avs2yuv to avs2pipemod. Any suggestion on corresponding avs2pipemod command line for this avs2yuv to imagemagick.

avs2yuv.exe -raw "script.avs" -o - | convert -limit memory 500MiB -limit map 1GiB -size 1920x1088 -depth 16 -define quantum:format=floating-point -define quantum:scale=65536.0 -set colorspace RGB -colorspace RGB rgb:- "%06d_output.tif"

Cheers.

sneaker_ger

3rd January 2012, 01:06

"-depth 16"
Does avs2pipemod support the high bit hacks that avs2yuv uses in the first place?

the_weirdo

3rd January 2012, 06:41

avs2pipemod supports raw output like avs2yuv too. I think here is the equivalent of Yellow_'s command line for avs2pipemod:
avs2pipemod.exe -rawvideo "script.avs" | convert -limit memory 500MiB -limit map 1GiB -size 1920x1088 -depth 16 -define quantum:format=floating-point -define quantum:scale=65536.0 -set colorspace RGB -colorspace RGB rgb:- "%06d_output.tif"

sneaker_ger

3rd January 2012, 07:34

avs2pipemod supports raw output like avs2yuv too.

You seem to be correct. I didn't got it working in my last tests, but it seems I just picked the wrong mode (stacked instead of interleaved) and wrongly assumed the piping tool would convert the high bitdepth into a different format.

Yellow_

3rd January 2012, 16:29

hey, thanks that's done the trick. :-)

SamKook

5th January 2012, 04:12

I was wondering, would it be possible to pipe the audio and video at the same time? To encode in XviD and MP3 with virtualdub without having to mux them afterwards for example.

Chikuzen

5th January 2012, 13:39

update

avs2pipemod-0.1.1.7z(removed)

I rewrote almost code.

* avs2pipemod detects avisynth version, and changes the actions.
　(thus, avs2pipe26mod was removed.)

* avisynth.lib is unnecessary any longer.
　for win64 builds can also be compiled now.

sneaker_ger

5th January 2012, 18:24

Thanks.
I noticed a "avs2pipe26mod.exe" is no longer included. Is this obsolete now?

the_weirdo

5th January 2012, 18:29

I think the answer is "Yes".

* avs2pipemod detect avisynth version and change actions.
(thus, avs2pipemod26 was removed.)

sneaker_ger

5th January 2012, 19:15

I'm blind, thx.

Chikuzen

6th January 2012, 11:56

I was wondering, would it be possible to pipe the audio and video at the same time? To encode in XviD and MP3 with virtualdub without having to mux them afterwards for example.

Your request means "I want a realtime streaming encoder only for uncompressed format."

Is it a very difficult subject ?

For present me, yes, it is very difficult...
Although I began to study the C language and one year has passed, it may be able to do, if I will try hard for three or more years.