avs2pipe

[doobry]

avs2pipe is a tool to output y4m video, wav audio, dump some info about the
input avs clip or suggest x264 blu-ray encoding settings.

Usage: avs2pipe [audio|video|info|x264] input.avs
audio - output wav extensible format audio to stdout.
video - output yuv4mpeg2 format video to stdout.
info - output information about aviscript clip.
x264bd - suggest x264 arguments for blu-ray disc encoding.


Download:

http://doobrymedia.com/avs2pipe-0.0.3.zip


It simply takes a path to an avs script that returns a clip with audio and/or
video and outputs it as a y4m (--video) or wav (--audio) stream to stdout. It
can also generate information about the clip (--info) in an easily parsable
format for gui's.

The advantage of avs2pipe over other solutions, other than it doing multiple
things in one and supporting more colorspaces is that it links directly with
avisynth.dll via the C interface, so it compiles with both Visual Studio and
MinGW and when used with wine in linux simply requires that the avisynth dll's
are in PATH, so no need to have vfw configured correctly. Also as it uses
WAVE_FORMAT_EXTENSIBLE for wav output and directly takes data from avisynth
it will output exactly the same data as avisynth uses internally.


Source is provided in the src folder and is released under the GPL-3.0.
In addition to the source is:

src\avisynth25 - header and link lib from AviSynth 2.5.8
src\avisynth26 - header from http://git.videolan.org/?p=x264.git;a=blob;f=extras/avisynth_c.h;hb=HEAD
link lib from AviSynth AVS 2.6.0 Alpha 2 [090927]


Projects and scripts to build the source are located in:

vs2010 - Project for Visual Studio 2010 Express
mingw - Batch file MinGW on Windows, sh script for MinGW under Linux


The YUV4MPEG2 output is inspired by Avs2YUV by Loren Merritt, it is basically
the same method, converted from C++ to C. The changes are:

Interlaced Support - Picks up the flags for interlaced content from
AssumeFieldBased(), AssumeTFF() etc. and will pass this
on to programs such as x264 eliminating the need to add
flags to the encoder command.

Extended Color Spaces - Thanks to Chikuzen when compiled against AviSynth 2.6
there is support for a whole host of extra colorspaces.


The WAV output is coded from scratch using specs from "the internet" and so
could be full of problems, altho I have not found any in testing yet.



Examples:

avs2pipe info input.avs
avs2pipe info input.avs > info.txt

avs2pipe video input.avs | x264 --stdin y4m - --output video.h264
avs2pipe audio input.avs | neroAacEnc -q 0.25 -if - -of audio.aac

avs2pipe audio input.avs > output.wav


Included Binaries:

avs2pipe_gcc.exe - compiled with MinGW gcc 4.5.2 on Windows
avs2pipe_vs.exe - compiled with Visual Studio 2010 Express cl 16.00.30319.01

avs2pipe26_gcc.exe - gcc compiled for AviSynth 2.6 Alpha 2
avs2pipe26_vs.exe - VS2010 compiled for AviSynth 2.6 Alpha 2


Changes:

Version 0.0.3 - Added Blu-Ray x264 command generator based on specs from
http://sites.google.com/site/x264bluray/
Added Support for new Colorspaces in AviSynth 2.6 with code
written by Chikuzen
Version 0.0.2 - Added Support for Interlaced Output, AssumeFieldBased etc.
Version 0.0.1 - Removed Progress Output for "Speed"
Pre-Alpha - Initial Release on doom10.org


Forum Links:

Original - http://doom10.org/index.php?topic=759
Doom9 Continuation - http://forum.doom9.org/showthread.php?t=160383

---

Hi,

Last year I wrote a little program called avs2pipe which combines avs2yuv and wav output into a single program.

The main advantage over the current solutions, and the reason I wrote it, is that it is in C and so can be compiled with gcc (mingw) under linux, or visual studio, and when used in a "wine" environment just requires the avisynth dlls to operate.

However I abandoned it as it worked for my task and no one really seemed that interested, but I just found people refering to it on google, so...

Is there anyone who would be interested in a newer build with the logging removed / optional to increase speed?

[Chikuzen]

Is there anyone who would be interested in a newer build with the logging removed / optional to increase speed?

Yes, of course :)

Because you did not reply to Underground78 (http://doom10.org/index.php?topic=759.msg5030#msg5030) for a long time, I thought that you had stopped that work.

[b66pak]

i am interested too..
_

[space1999]

I am interested too (+1) :)

BTW, does it already support channel-mapping AND "100%-compliant" Wave-Format-Extensible info? Because sometimes I have to use sox.exe for correcting the outputs from wavi.exe...

[smok3]

would be interested in ubuntu and especially OSX version.

[doobry]

ok i'll pull out the sources from my dead projects and get it checked and uploaded tomorrow.

as for the channel mapping, at the moment it doesn't remap any channels, but i wrote the wav output from scratch using the specs I found on the internet, so it should be "100%-compliant Wave-Format-Extensible", the urls for the specs should be in comments in the source, so if i can work out channel mapping it will not be hard to add it, the main challenge would be parsing the command line options, as the current method simply matches the first argument.

as for ubuntu and mac os x, assuming you have a working wine install, it should just work... you would only need mingw if you wanted to compile it yourself.

[tebasuna51]

A program like this don't need make channel mapping.
The channel mapping is a encoder/decoder job. There are enough AviSinth audio decoders than work fine.

@space1999
wavi works fine, maybe is a decoder problem, can you put some sample.

[space1999]

@tebasuna51

This is really weird, but I swear, some weeks ago, Media Player Classic refused to play a 3.0 WAV file written by wavi, but today, the same file is accepted fine,
:confused: :confused: :confused:

Anyway, here goes an example of the difference between the WAVs outputted by wavi and the WAVs rewritten by sox:

from wavi:

52 49 46 46 3C 61 52 01 57 41 56 45 66 6D 74 20
28 00 00 00 FE FF 03 00 80 BB 00 00 00 65 04 00
06 00 10 00 16 00 10 00 07 00 00 00 01 00 00 00
00 00 10 00 00 80 AA 00 00 38 71 9B 64 61 74 61
00 61 52 01 00 00 00 00 00 00 00 00 00 00 00 00

from sox:

52 49 46 46 3C 61 52 01 57 41 56 45 66 6D 74 20
28 00 00 00 FE FF 03 00 80 BB 00 00 00 65 04 00
06 00 10 00 16 00 10 00 00 00 00 00 01 00 00 00
00 00 10 00 80 00 00 AA 00 38 9B 71 66 61 63 74
04 00 00 00 80 65 38 00 64 61 74 61 00 61 52 01

[doobry]

@tebasuna51

the initial problem that I had was that all the audio 2 pipe type programs for avisynth did not "work fine", all the ones I could find were just vfw programs, or are written in .NET etc, which for me under wine was causing problems.

avs2pipe directly links with the avisynth.dll which solved me a lot of problems under linux and gives direct access to any avisynth stuff. As far as I can tell wavi etc just use vfw and so have no real idea about avisynth at all.

as for the channel mask, the support is "already" in there, it just sets the mask to 0 at the moment, so i could add wavi style support of just specifying the mask as an int at the command line, but that seems a little user unfriendly. a suggestion for how it should work from a users point of view is welcome. i could for example let people specify a variable in the avs script and pull it out.


@space1999

one of them programs is using a different endianness to the other for the subformat waveex field, which means one of them probably has a mistake in it.

next post should be the same thing as was on doom10, but without progress reporting...

[doobry]

Here is the zip file with the source, updated visual studio project files, a batch script to build it on windows using mingw and two pre built binaries...

http://doobrymedia.com/avs2pipe-0.0.1.zip

Comments and suggestions are more than welcome.

@tebasuna51

if you could have a look at the source for anything stupid I'd be grateful, the wav handling is mostly in wave.c and wave.h and is pretty heavily commented as I was trying to work it all out from the specs.

[doobry]

from wavi:

52 49 46 46 3C 61 52 01 57 41 56 45 66 6D 74 20
28 00 00 00 FE FF 03 00 80 BB 00 00 00 65 04 00
06 00 10 00 16 00 10 00 07 00 00 00 01 00 00 00
00 00 10 00 00 80 AA 00 00 38 71 9B 64 61 74 61
00 61 52 01 00 00 00 00 00 00 00 00 00 00 00 00

from sox:

52 49 46 46 3C 61 52 01 57 41 56 45 66 6D 74 20
28 00 00 00 FE FF 03 00 80 BB 00 00 00 65 04 00
06 00 10 00 16 00 10 00 00 00 00 00 01 00 00 00
00 00 10 00 80 00 00 AA 00 38 9B 71 66 61 63 74
04 00 00 00 80 65 38 00 64 61 74 61 00 61 52 01

Had a little look into this, and avs2pipe matches the endianess of sox, (80 00 00 AA...) so i'm guessing that there is a problem with wavi (00 80 AA 00...), or that myself and sox got it wrong.

Grabbed a sample from http://www-mmsp.ece.mcgill.ca/documents/audioformats/WAVE/Samples.html and it matches sox and myself, the endianess I think can be architecture specific, but as everything is running on x86/x64 I believe they should match.

[Chikuzen]

It seems that ver 0.0.1 is only support YUV420 on video.
Do you have any schedule for supporting other colorspaces (YUV422/YUV444) and latest YUV4MPEG2 format?

[doobry]

@Chikuzen

I have no schedule for supporting other colourspaces, as I have nothing to test against, i.e. samples, encoders, players, and have not seen the YUV4MPEG2 specs. However I'm more than willing to have a go at adding it if someone comes up with a way for me to test it, and some specs.

At the moment it simply adds a ConvertToYV12 filter to anything that is not YV12.

FYI, a quick google turned up http://wiki.multimedia.cx/index.php?title=YUV4MPEG2 is that what you mean by the new format?

Update: I've had a play and avisynth 2.58 doesn't support YUV422/YUV444 so I assume you are using 2.6, and if so where did you get it, and can you give me a link to the sources.

[Chikuzen]

FYI, a quick google turned up http://wiki.multimedia.cx/index.php?title=YUV4MPEG2 is that what you mean by the new format?

I think that latest format is this (http://linux.die.net/man/5/yuv4mpeg).
and you will be able to test about YUV422 by using rawsource.dll .
http://forum.doom9.org/showthread.php?p=1470266#post1470266

[doobry]

@Chikuzen

What version of avisynth are you using? and if its 2.6 can you point me to the source code.

[Chikuzen]

I'm using 2.6.0alpha2(x86) now. http://forum.doom9.org/showthread.php?t=149113
but that RawSource.dll is for 2.5.
2.6.0alpha is completely compatible with 2.5.6 or later.

If you want to develop for 2.6 or later, then see manao's code.
http://forum.doom9.org/showpost.php?p=1411605&postcount=473
it includes 2.6 header file.

[doobry]

From what I can tell avisynth 2.5 does not support YUV422/YUV444 colourspaces, so if i add support for them it will have to be for 2.6 only.

Had a look at the rawsource source code, and it seems to "mangle" for want of a better word the input into one of the 2.5 supported formats.

I'm not sure exactly what your requirement is, I assume you want to be able to output YUV422 video from avisynth and have avs2pipe provide it as YUV4MPEG2 output without any conversion? If that is the case could you provide me with a small sample of your input YUV4MPEG2 and avs script so I can have a play with it.

Thanks for the header link, but I need the c header avisynth_c.h and the .lib to link with, really I'd like to track down the cvs people are using, but the sourceforge one seems stale, and it seems 2.6 is mostly passed about via forum links to random hosts, which is not good if i'm trying to develop against it.

[Chikuzen]

ah, i understood what you say.
I must study how to write the code that convert YUY2 to Planar YUV422.

[doobry]

I'm pretty much willing to add any support that people want, provided I can do it, all I really need is a test case and samples so I can make sure it works.

Personally I don't have any current use for anything other than 420 as I've only got DVD / BluRay sources, but I can see how it could be useful, so I'm willing to stick it on the todo list and add some macro's to enable 2.5 & 2.6 builds. Altho in this case I'm kinda stuck without an updated C interface that has support for the colourspaces.

However this has all caused me to add interlaced video support to the YUV4MPEG2 header, which I'm testing now, means I can convert interlaced mpeg2 to interlaced h264 without specifing extra options to x264.

PS. When you learn how to "convert YUY2 to Planar YUV422" please share the info with me, I'm clueless on these formats.

[Gavino]

Thanks for the header link, but I need the c header avisynth_c.h and the .lib to link with, really I'd like to track down the cvs people are using, but the sourceforge one seems stale
Do you mean this? It looks active to me.
http://avisynth2.cvs.sourceforge.net/viewvc/avisynth2/avisynth/

and it seems 2.6 is mostly passed about via forum links to random hosts, which is not good if i'm trying to develop against it.
I don't know what forum links you are referring to.
Chikuzen posted the link to the current 2.6 release (see above).
Perhaps you are thinking about the various unofficial MT builds floating around.

[ganymede]

Is it possible to combine audio and video output ? If I could pass audio and video at the same time to mencoder or ffmpeg, it would be a big step forward for me, compared to avs2yuv.

[doobry]

@ganymede

There is no reason not to be able to combine the outputs provided there is a simple container format that can wrap the video and audio unmangled, that the target tool can accept via stdin, any suggestions?

[doobry]

@Gavino

What I mean by stale is that the latest release is from 2009, and I cannot find an updated avisynth_c.h header in the cvs that supports the new colourspaces I think 2.6 has.

And yes I am thinking of the unofficial builds, I'm new to all this, and I don't want to write something just to find out that everyone is actually using something different or that the work had already been done.

[Chikuzen]

oh doobry, did you think that I'm using 2.6.0MT :eek:
Then, it is a misunderstanding.
IMO, official 2.6.0alpha is more stable and faster than 2.5.8(not MT).
(I can't understand why IanB doesn't change alpha to latest stable :p)

[doobry]

doesn't really matter what version you are using at this stage, I cannot find an updated avisynth_c.h that contains support for the new colourspaces, so I'm going to leave it until I can find an "official" source.

[Gavino]

What I mean by stale is that the latest release is from 2009, and I cannot find an updated avisynth_c.h header in the cvs that supports the new colourspaces I think 2.6 has.
I think this has something to do with developments regarding 'baked code' - see this post.
Perhaps IanB can shed more light on the subject.

[doobry]

@Gavino

Thats exactly what I'm talking about, thank you!!! been unable to find that info myself, will give it a good read when I get some time.

[doobry]

I've uploaded version 0.0.2 which has added interlaced support, and adds a dump of avisynth's internal pixel_type field to the --info option, as I thought it might be of use to be able to see / post the value avisynth is presenting to the program.

http://doobrymedia.com/avs2pipe-0.0.2.zip

[Chikuzen]

thanks for updating.

BWT, there is a typo in line 37 of readme.txt
s/aac/mp4/g

[tebasuna51]

About Wavi.

Please read the pending bug http://sourceforge.net/tracker/?func=detail&aid=2957741&group_id=196137&atid=956250
From this page you can download the wavi.exe fixed.

[doobry]

@tebasuna51, that looks correct, i found that link to samples I posted earlier great for testing if things were working correctly by piping them though my code and checking they came out the same.

fyi can you confirm that wavi handles channel mapping my parsing an int from the command line and directly writing it into the header? From a brief look at the code that seems to be what it does, but I just wanted to make sure.

[kemuri-_9]

you can find the modified avisynth_c.h that x264cli uses here (http://git.videolan.org/?p=x264.git;a=blob;f=extras/avisynth_c.h;hb=HEAD)

which is good enough to support recognizing the newer 2.6 csps if that's all you need.
it's not complete enough to write a filter with to support the new csps however.

[ganymede]

There is no reason not to be able to combine the outputs provided there is a simple container format that can wrap the video and audio unmangled, that the target tool can accept via stdin, any suggestions?Good news. But are you sure that a container is needed, and that both audio and video cannot be output together as a raw data stream ?
It seems that ffmpeg and mplayer/mencoder can work with a raw data stream where audio and video are "mixed" (I don't know if it's the right word). For instance, I tested the following commands on my linux system :cat file | ffplay -
cat file | mplayer -
"file" was either in .avi, .mp4 or .flv container, and in all cases ffplay or mplayer were able to read it, with both audio and video (so I guess ffmpeg and mencoder would be able to process it).

[doobry]

.avi, .mp4 or .flv container

what you are doing works because the avi etc files are sent directly to ffplay etc, they are already containers... something is needed to tell the receiving application what is video, what is audio, and how they are encoded. The current version provides YV12 video in a YUV4MPEG2 "container" and PCM / FLOAT audio in WAV container. In theory it should be possible to say use an AVI container to send both, but that assumes that you can send raw YV12 data in an avi file stream written on the fly, ideally one frame at a time.

Basically I've not looked into it, but I can easily make it output both streams, the issue is doing it in a way that other programs can understand.

[ganymede]

what you are doing works because the avi etc files are sent directly to ffplay etc, they are already containers...Oops, you're right :rolleyes:
If it's not too complicated to embed YV12 video and PCM audio streams in an avi container, I guess it would do the trick.

[doobry]

I really should be in bed, but I've had a quick google and YV12 is a supported microsoft / apple FOURCC, so it should be possible, the only problems I see is the receiving program being willing to play the avi without random access, which should be ok if you can cat then in, and it ignoring the 4GB limit that avi's 32bit headers impose. The same limit applies to WAV files but the encoders I use ignore the header so work fine.

Will have a go next week.

[space1999]

@ tebasuna51:

thanks for the unb0rked :) build of wavi.exe

@ doobry:

if a container is really desirable/required, would be a big problem to use Matroska ? AFAIK MP4 does not support uncompressed audio (not "natively" anyway), and even though AVI currently has no filesize limit, one can never be "100%-sure" of which applications fully respect the OpenDML specs.

[ganymede]

@doobry : thank you very much.

[tebasuna51]

@tebasuna51,...
fyi can you confirm that wavi handles channel mapping my parsing an int from the command line and directly writing it into the header? From a brief look at the code that seems to be what it does, but I just wanted to make sure.

I'm not sure what ask you.

If is a wavi test, yes I make many test (because part of the code is from me) and I think work fine.

If is about your soft seems also work (gcc build tested), but:

- A minor info problem, the code seems ok:
avs2pipe [info]: writing 20 seconds of 0 Hz, 48000 channel audio.

- The Chunk 'fact' is not needed with WAVE_FORMAT_PCM/WAVE_FORMAT_IEEE_FLOAT.
The 'samples' data is always 'data_length'/'BlockAlign'

- The WAVE_FORMAT_EXTENSIBLE_HEADER is useless without a correct MaskChannel value.
AviSynth don't have a MaskChannel audio property, then decoders like NicAudio, than know the proper MaskChannel value, can't pass it to AviSynth.
And AviSynth can't pass the value to wavi/avs2pipe.
Wavi put a default value for each numchannels and accept a parameter to change the default.

[doobry]

A minor info problem, the code seems ok:

Thanks will try to work that out, clearly something is not right.


The Chunk 'fact' is not needed with WAVE_FORMAT_PCM/WAVE_FORMAT_IEEE_FLOAT.

I saw that it was not required with WAVE_FORMAT_PCM but the info i had was a little unclear on WAVE_FORMAT_IEEE_FLOAT, i.e. info said it was "needed when the samples are not WAVE_FORMAT_PCM", so I figured as it is allowed, Its best to include it just incase... altho it should never be needed.


The WAVE_FORMAT_EXTENSIBLE_HEADER is useless without a correct MaskChannel value.

It was my understanding that the spec included an "all zero" option for the mask that means the channels are mapped in order, ie first channel -> first output etc. As you said I cannot get a channel mask from avisynth, so I figured using this "blank" mask was the best option until I add a way to specify a mapping.

[Chikuzen]

ok, It seems that I succeeded in modifying avs2pipe for support other csps (require avisynth2.6.0α2) :D

avs2pipe260-0.0.2 (http://www.mediafire.com/download.php?glbg875ch4hx81q)
*Added support YUV444(YV24) / YUV422(YUY2/YV16) / YUV411 / Y8 colorspace Y4M output.

thanks to doobry and kemuri-_9 !

[ganymede]

(require avisynth2.6.0α2)It works for me (using wine on linux) with avisynth2.6.0α2, but not SEt's 2.6 MT. With 2.6 MT, output is empty (no data). Nice step forward, though.

[doobry]

ok, It seems that I succeeded in modifying avs2pipe for support other csps (require avisynth2.6.0α2) :D

avs2pipe260-0.0.2 (http://www.mediafire.com/download.php?glbg875ch4hx81q)
*Added support YUV444(YV24) / YUV422(YUY2/YV16) / YUV411 / Y8 colorspace Y4M output.

thanks to doobry and kemuri-_9 !

Yey that is fan fricking tastic, I totally missed the link from kemuri-_9 to that darned header I couldn't find, and now the work is basically done for me :D

Will try to roll the changes into my "master" version, can hopefully stick in a macro to enable support if requested, and slap your name in the comments.

@Chikuzen, where did you get the avisynth.lib link library? is it the one that i included in the original zip (2.5.8) or one from avisynth2.6.0α2? as I think SEt's 2.6 MT build did not include the link library, and I cannot find his source release for it...

[Chikuzen]

@Chikuzen, where did you get the avisynth.lib link library? is it the one that i included in the original zip (2.5.8) or one from avisynth2.6.0α2? as I think SEt's 2.6 MT build did not include the link library, and I cannot find his source release for it...

I got it from official 2.6.0α2 binary(extracted installer with 7zip).

I don't know where 2.6MT source code is because I lost the interest in MT after SEt disappears .

[doobry]

Well my version now in theory supports the new colorspaces, thanks to the header from kemuri-_9, so there are now 4 binaries, gcc and vs compiled against 2.5.8 and 2.6a2, just need something to test them with.

As for 2.6MT support, I am not going to bother, the latest source link I could find was just above a post detailing a bunch of bugs, so it seems I cannot get a link library for the MT version, and as something weird is clearly happening with it I'm not going to waste my time without the proper sources and libs.

Also coming in version 0.0.3, Blu-Ray encoding suggestions for x264, it looks at your avs script and picks the correct settings from http://sites.google.com/site/x264bluray/ to pass to x264... kinda pointless for most people but it lets me define everything in the avs file as I use batch files to encode.

[doobry]

New Version 0.0.3

http://doobrymedia.com/avs2pipe-0.0.3.zip

Changed the command line usage to remove "--" from the command arguments, as there can be only one command and if I add "options" later this will keep things neater.

Usage: avs2pipe [audio|video|info|x264] input.avs
audio - output wav extensible format audio to stdout.
video - output yuv4mpeg2 format video to stdout.
info - output information about aviscript clip.
x264bd - suggest x264 arguments for blu-ray disc encoding.

Added Blu-Ray x264 command generator based on specs from http://sites.google.com/site/x264bluray/

Added Support for new Colorspaces in AviSynth 2.6 with code written by Chikuzen

I messed around a little with how the colorspaces are handled to make it a bit neater for the macro to switch between 2.5 / 2.6 support, so their could be bugs as its not tested fully.

I do know that the 2.6 versions will prob hang under 2.5 and the 2.5 versions will hang under 2.6 if you try to use new features, but thats expected.

[ganymede]

I just tested avs2pipe26_gcc.exe with SEt's 2.6 MT under linux/wine... and it works ! Don't ask me why it does :p , but it's a good news for me.
Thank you guys for your work :thanks:

[doobry]

Speaking of weirdness... why does outputting both an I420 and YV12 colorspace result in exactly the same data... shouldn't AVS_CS_I420 be y-u-v planar, while AVS_CS_YV12 is y-v-u planar? As I'd coppied the method from avs2yuv I hadn't really spent too much time looking colorspace details, but thanks to Chikuzen I had to fire up google and look, and from what I found in theory the code should not have worked from day one as the u and v planes should have been swapped (its called YUV4MPEG2 and not YVU4MPEG2 afterall).

The avs2pipe code takes each plane and writes then out in the order that avisynth presents them... litterally a planes 1-3 for loop (well 0-2), given this shouldn't streams marked as AVS_CS_I420 give different output to AVS_CS_YV12? It would not be hard to swap the planes if needed, and it should be, but it seems that it "just works (tm)" when it really shouldn't.

So anyone with a bit more AviSynth knowledge know whats going on? Does AviSynth store all planar colorspaces in the same order in memory (y-u-v) and just set the pixel_type for reference of where its reordered data came from?

PS. The current download version (0.0.3) automatically adds a ConvertToYV12 filter to anything it doesn't understand, which includes AVS_CS_I420, obviously I've turned that off for AVS_CS_I420 on my current test version and told it to handle it exactly the same as AVS_CS_YV12.

[Gavino]

The avs2pipe code takes each plane and writes then out in the order that avisynth presents them... litterally a planes 1-3 for loop (well 0-2), given this shouldn't streams marked as AVS_CS_I420 give different output to AVS_CS_YV12? It would not be hard to swap the planes if needed, and it should be, but it seems that it "just works (tm)" when it really shouldn't.

So anyone with a bit more AviSynth knowledge know whats going on? Does AviSynth store all planar colorspaces in the same order in memory (y-u-v) and just set the pixel_type for reference of where its reordered data came from?
The planes (Y, U, V) are always numbered the same, independently of the order they are stored in memory, so if you always ask for them (via GetReadPtr()) in a given order, that's the order you will get.

However, I'm confused by your saying planes 0-2, since Y, U, V are numbered 1, 2 and 4.
enum {
PLANAR_Y=1<<0,
PLANAR_U=1<<1,
PLANAR_V=1<<2,
...

[doobry]

I'm a giant idiot... ofc they are requested by name (or at least a named constant), ffs, I have an array at the top to map the 0-2 from the for loop to the constants AVS_PLANAR_Y, AVS_PLANAR_U, AVS_PLANAR_V... so that the loop is simpler.

So basically AviSynth isn't braindead, and if you request AVS_PLANAR_U it gives you U no matter what the formats storage order may be...

This is what happens when you convert someone else's method, to ms's ancient c that requires all variables declared at the beginning of the functions body, then don't look at it for 6 months.

@Gavino thanks for the well aimed cluebat. PS GScript and GRunT look pretty fantastic, would you have any issues with me adding auto loading of scripts via GImport? I can see two easy ways, a) Just load all scripts via GImport if the function / dll is present, or b) trying to load it if the passed file is *.gscript (or similar). Do you have a preference?

I.E. You would be able to do avs2pipe video myscript.gscript and have it autoload the plugin and GImport the script...

[Gavino]

would you have any issues with me adding auto loading of scripts via GImport? I can see two easy ways, a) Just load all scripts via GImport if the function / dll is present, or b) trying to load it if the passed file is *.gscript (or similar). Do you have a preference?

I.E. You would be able to do avs2pipe video myscript.gscript and have it autoload the plugin and GImport the script...
Sounds like a cool idea either way. I don't have a strong preference, but if going for option b, how about .avsg as the extension (based on existing use of .avsi)?

[doobry]

using an avsg extension would be my preference, as it means that scripts with GScript features would be harder to mistake for "normal" avs scripts, either linked on forums or "found" years after they were written.

[doobry]

avs2pipe [info]: writing 20 seconds of 0 Hz, 48000 channel audio.

I went through and changed a bunch of the standard fprintf placeholders to the super special Microsoft versions in the last release (0.0.3), are you still having this problem as I cannot reproduce it?

If its still there what flavor (vs or gcc etc) and type of input are you using?

[tebasuna51]

I went through and changed a bunch of the standard fprintf placeholders to the super special Microsoft versions in the last release (0.0.3), are you still having this problem as I cannot reproduce it?

Is ok now, thanks.

[IanB]

@doobry,

In terms of plane order, generally you cannot assume a particular order in memory. In all places except env->NewVideoFrame(vi), CS_YV12 and CS_I420 are considered identical. This distinction is to allow source filters to import YV12 and I420 video data directly into a PVideoFrame without needing to blit the individual planes.

[doobry]

In terms of plane order, generally you cannot assume a particular order in memory. In all places except env->NewVideoFrame(vi), CS_YV12 and CS_I420 are considered identical. This distinction is to allow source filters to import YV12 and I420 video data directly into a PVideoFrame without needing to blit the individual planes.

Given this does reading via avs_get_read_ptr_p(frame, planes[p]) etc, always return the correct pointer to Y, U, V plane when using the AVS_PLANAR_Y, AVS_PLANAR_U, AVS_PLANAR_V constants? Assuming that another filter has not messed something up?

i.e. Is avs2pipe doing it right? The only testing I can do easily is using FFMpegSource which marks the stream I420, and DGIndex which marks the same source as YV12... Both given identical output with avs2pipe, using the same YUV4MPEG2 generation method, so it works for me, but I just want to make sure its not a fluke.

[Gavino]

Given this does reading via avs_get_read_ptr_p(frame, planes[p]) etc, always return the correct pointer to Y, U, V plane when using the AVS_PLANAR_Y, AVS_PLANAR_U, AVS_PLANAR_V constants? Assuming that another filter has not messed something up?
Yes, because GetReadPtr (or avs_get_read_ptr_p in the C API) uses the appropriate offset (offsetU, offsetV) from the frame header (previously stored by NewVideoFrame(vi)).

[IanB]

Given this does reading via avs_get_read_ptr_p(frame, planes[p]) etc, always return the correct pointer to Y, U, V plane when using the AVS_PLANAR_Y, AVS_PLANAR_U, AVS_PLANAR_V constants? Assuming that another filter has not messed something up?

i.e. Is avs2pipe doing it right? The only testing I can do easily is using FFMpegSource which marks the stream I420, and DGIndex which marks the same source as YV12... Both given identical output with avs2pipe, using the same YUV4MPEG2 generation method, so it works for me, but I just want to make sure its not a fluke.
For testing SwapUV() with a planar just flips the pointers in the header (zero cost), so you can make sure you code is plane order agnostic.

[doobry]

Thanks to IanB and Gavino causing me to look though the color space handling code, I've rewritten it for the "next" version. It now uses an internal buffer to organize each frame, before writing it to stdout with a single call to fwrite. Before it called fprintf for the FRAME header, and then fwrite for each row of data...

This has given me a huge speed boost, ~1.3 seconds to write 1000 frames to null, vs ~2.1 seconds using the old method, however I noticed that it will no longer output to the windows console, not a problem as you should never want to do that, but it did make me wonder if there is a limit to how much you can write to a pipe with a single fwrite... it still works fine via x264 or piping to a file, but I want to be sure.

So before I upload this and remove the link to the working version, does anyone want to have a test, or have any views on this... I will provide a download link via PM to a test version, but I'm not posting it here as I've noticed scrapers are linking to my downloads, which I don't really mind but I don't want some random site posting a possibly broken version without the info saying it might be busted.

Obviously this "huge" speed increase will be pretty insignificant vs the slowdown caused by say piping the output to x264 but it all helps.

[Chikuzen]

@doobry
It seems that x264 team schedules to change some encode options.
https://github.com/DarkShikari/x264-devel/commit/4a5f4432172b33bfaef1e2f6c63c468060efe1e4

I think that you might discontinue "x264bd" because it will be outdated soon.

[doobry]

@doobry
It seems that x264 team schedules to change some encode options.
https://github.com/DarkShikari/x264-devel/commit/4a5f4432172b33bfaef1e2f6c63c468060efe1e4

I think that you might discontinue "x264bd" because it will be outdated soon.

From that commit link...

This option doesn't automatically enforce every aspect of Blu-ray compatibility (e.g. resolution, framerate, level, etc).

As I said when I added it, its really a feature for me, so that I don't have to refer to the website and lookup the correct options for the file i'm encoding... It does however error out when it cannot find a matching "profile" for the video, something that the commit comment says the changes do not do.

It would be great if the same feature ended up in x264 as it would mean I can be 100% sure i'm doing the right thing, x264bd is just a consolidation of info on the web, for my use. Bring on the day I can remove it can have a --profile bluray option on x264 that checks everything (framerate, resolution, interlacing etc) will be correct.

And if someone is relying on it, then I'll just modify the options to match the new guide... when someone writes one :D

[doobry]

Did a little benchmarking, real time in seconds, run 3 times each.

avs2pipe 0.0.3:

avs2pipe26_vs 54.23 54.06 54.01
avs2pipe26_gcc 51.83 52.60 52.10
avs2pipe_vs 54.35 54.27 54.26
avs2pipe_gcc 51.86 52.10 51.97


avs2pipe 0.0.4 alpha:

avs2pipe26_vs 32.99 32.73 32.84
avs2pipe26_gcc 32.85 32.72 32.82
avs2pipe_vs 32.81 32.84 32.95
avs2pipe_gcc 32.80 32.73 32.76


avs2yuv 0.24:

downloaded 45.91 46.43 46.11
vs2010 express 54.20 54.44 54.00


Test Was:

Windows 7 x64, Quad Core AMD, AviSynth 2.6 Alpha 2
32374 frames 720x576 YUV420 video from PAL DVD
FFMpegSource for m2v file
FFMpegSource for ac3 file (Not Processed)
Output piped to NUL
MSYS bash time used to time execution
MSYS md5sum used once to check output identical


Thing that really surprised me was that opening the avs2yuv project in Visual Studio 2010 Express and compiling it reduced it speed from around 46sec to 54sec... Is the Express compiler really that bad?

[Chikuzen]

Hi.
I fiddled with the code of this tool for about one month.
(The purpose is to make it to more wieldy thing for me, and learning C language.)

binary
deleted

source code
https://github.com/chikuzen/avs2pipemod

Differences with original
* Display total elapsed time.
* contents of 'info'
* New option 'benchmark'.
* 'y4mp', 'y4mt', 'y4mb' and 'rawvideo' options instead of 'video'.
* Add SAR info to yuv4mpeg2 stream header.
* FieldBased input will be corrected to framebased on yuv4mpeg2 output mode.
* Colorspace conversion that takes colormatrix and interlace into consideration.
* 'rawaudio' option.
* Convert bit depth function in audio output mode.
* Only the compilation with mingw/msys is supported.

[b66pak]

thanks a lot...
_

[Chikuzen]

updated.

avs2pipemod-20110510.zip (http://www.mediafire.com/download.php?pl0lx57i7dqsin7)

*Fix incorrect output of 'x264bd', and some modified.

EDIT
Sorry, I found a bug in '-audio' and '-rawaudio'.
Please Download again if you are using avs2pipemod-20110509.zip

[Midzuki]

@ Chikuzen:

Many many thanks :thanks: :thanks: :thanks:

BTW, ¿ where is doobry ? :)

[zambelli]

Thanks for this tool, Doobry and Chikuzen. Exactly what I was looking for, and perfect timing too. :)

[Chikuzen]

update

avs2pipemod-20110703.zip (http://www.mediafire.com/download.php?aa43daakid35kaa)

add new options '-wav' and '-extwav', and remove '-audio'.

'-wav' outputs audio with normal riff-wave header.
'-extwav' outputs audio with wave extensible header containing channel-mask.

[Midzuki]

:thanks: :thanks: :thanks: :thanks: :thanks:

[IanB]

add new options '-wav' and '-wavext', and remove '-audio'.
Disappointed you don't honour legacy options, now everybody with -audio has to edit their junk to -wav. :(

But still a great project.

[Chikuzen]

Disappointed you don't honour legacy options, now everybody with -audio has to edit their junk to -wav. :(.

no, I changed -audio to -extwav :p

However, I changed it again because what you said was justifiable.
avs2pipemod-20110703-2.zip (http://www.mediafire.com/download.php?9x0du58bii28t27)

@@ -720,6 +720,7 @@ void parse_opts(int argc, char **argv, params *params)
struct option long_opts[] = {
{"wav", optional_argument, NULL, 'w'},
{"extwav", optional_argument, NULL, 'e'},
+ {"audio", optional_argument, NULL, 'e'},
{"rawaudio", optional_argument, NULL, 'a'},
{"y4mp", optional_argument, NULL, 'p'},
{"y4mt", optional_argument, NULL, 't'},

[IanB]

Thanks! :D

[henryho_hk]

Is this build multithread-enabled? I have a AMD 1090T X6 and tried:

avs2pip26mod.exe -y4mp some-720p-with-multithread.avs | x264.exe .... --output video.h264 --stdin y4m -
x264.exe .... --output video.h264 some-720p-with-multithread.avs

The first command can only keep one core busy (~16-17%) and the second one can make it ~99%.

[Chikuzen]

Is this build multithread-enabled? I have a AMD 1090T X6 and tried:

avs2pip26mod.exe -y4mp some-720p-with-multithread.avs | x264.exe .... --output video.h264 --stdin y4m -
x264.exe .... --output video.h264 some-720p-with-multithread.avs

The first command can only keep one core busy (~16-17%) and the second one can make it ~99%.

If you want to use avs2pipe(mod) with avisynthMT, add Distributor() at the end of your script.

[CarlEdman]

Is this build multithread-enabled? I have a AMD 1090T X6 and tried:

avs2pip26mod.exe -y4mp some-720p-with-multithread.avs | x264.exe .... --output video.h264 --stdin y4m -
x264.exe .... --output video.h264 some-720p-with-multithread.avs

The first command can only keep one core busy (~16-17%) and the second one can make it ~99%.

Avs2pipe(26)?(mod)? does not need to be MT enabled--all of that is determined by the avisynth library, invoked by both avs2pipe and x264. And the scripts need a "Distributor()" at the end to use multi-threading (a long time ago that was not the case for x264, but it was necessary for quite a while--I have not checked recently if that requirement has gone away again).

So why is one execution multithreaded and the other not? One possibility is that you are using 64-bit x264. Avs2pipe is 32-bit. So if you have, a non-multithreaded 32-bit version 2.6 avisynth.dll installed, but a 64-bit multithreaded avisynth.dll (that is quite possible--the 32-bit and 64-bit system look for the same dll file, but in different paths), then you'd see just what you observe.

[Gavino]

So why is one execution multithreaded and the other not? One possibility is that you are using 64-bit x264. Avs2pipe is 32-bit. So if you have, a non-multithreaded 32-bit version 2.6 avisynth.dll installed, but a 64-bit multithreaded avisynth.dll (that is quite possible--the 32-bit and 64-bit system look for the same dll file, but in different paths), then you'd see just what you observe.
The reason is simply that x264 is 'MT-aware' as far as Avisynth is concerned - if it detects that you are running a MT version of Avisynth (and have used SetMTMode), it will make the necessary call to Distributor() itself.

avs2pipe, on the other hand, is not 'MT-aware' so does not call Distributor(), resulting in single threaded running.
(which is why you have to add it yourself.)

Note that for applications using Avisynth via the VfW interface (eg VirtualDub), this is not an issue since in MT Avisynth this path automatically invokes Distributor().

[Chikuzen]

Why do you want 64bit version ?
There are always avs2yuv and wavi for x64.

[Chikuzen]

then, use wavi.
http://forum.doom9.org/showpost.php?p=1527102&postcount=4

[Chikuzen]

I don't understand how to use it. avs2pipemod works fine for me:
avs2pipemod.exe -rawvideo %avs% | C:\Soft\Video\bin\x264-10bit.exe
avs2pipemod.exe -extwav=16bit %avs% > %wav%

If you don't need channel-mask, then
wavi input.avs output.wav
If you need channel-mask, then
wavi input.avs output.wav /x
If you want to output rawvideo, then
avs2yuv input.avs -raw - | x264 - --demuxer raw ...

FYI, currently, it requires avisynth.lib to compile avs2pipe(mod).
However, there is no avisynth.lib for x64.
Thus, it is necessary to rewrite many codes.

[Chikuzen]

wavi is a program that use vfw interface like VirtualDub/avs2avi/etc...
If wavi cannot treat avs as avi, you will have failed in installation of avisynth.

[Chikuzen]

update.

avs2pipemod-20110919.zip (http://www.mediafire.com/download.php?p2iyweklw0tp76a)

*add new option 'trim'

[SamKook]

Do you still plan on releasing v0.0.4? I'd love to get a similar bump in speed I see with your alpha version benchmark since using this is the only way I found to prevent crashing with memory heavy filters and weird things happen when I use avs2yuv. Also, having everything all in one is pretty great.

[doobry]

Do you still plan on releasing v0.0.4? I'd love to get a similar bump in speed I see with your alpha version benchmark since using this is the only way I found to prevent crashing with memory heavy filters and weird things happen when I use avs2yuv. Also, having everything all in one is pretty great.

The source code is here, there is no tagged 0.0.4 release or binaries, but building it isn't hard

https://github.com/doobry/avs2pipe

[Chikuzen]

oh doobry! welcome back!

[CarlEdman]

Ditto to the Welcome Back, doobry.

But could you please make a tagged release with binaries of 0.04? Building binaries is something of a hassle for those without a Windows dev environment.

[Chikuzen]

update

avs2pipemod-20111230.zip
(http://www.mediafire.com/download.php?uw2z0pzax1dqnf8)

*port video output processing from latest avs2pipe.

[b66pak]

thanks a lot...
_

[CarlEdman]

:thanks:

[SamKook]

Thanks doobry.

It's indeed very easy to compile; change the path to MinGW in the .bat, run it... and you're done.
It compiled so fast that I ran it twice since I was sure it had failed the first time.

And I got a 3.12% speedup in my first test so that's a good thing.

Edit: The second test when encoding them back to back in similar condition is less good though:
h264_10bit_a2p_0-0-3: 0:17:44.77
h264_10bit_a2p_0-0-4: 0:17:42.56
h264_10bit_a2pmod_0-0-3: 0:17:44.16
h264_10bit_a2pmod_0-0-4: 0:17:38.85

[Yellow_]

hi

I'm struggling to migrate from avs2yuv to avs2pipemod. Any suggestion on corresponding avs2pipemod command line for this avs2yuv to imagemagick.

avs2yuv.exe -raw "script.avs" -o - | convert -limit memory 500MiB -limit map 1GiB -size 1920x1088 -depth 16 -define quantum:format=floating-point -define quantum:scale=65536.0 -set colorspace RGB -colorspace RGB rgb:- "%06d_output.tif"


Cheers.

[sneaker_ger]

"-depth 16"
Does avs2pipemod support the high bit hacks that avs2yuv uses in the first place?

[the_weirdo]

avs2pipemod supports raw output like avs2yuv too. I think here is the equivalent of Yellow_'s command line for avs2pipemod:
avs2pipemod.exe -rawvideo "script.avs" | convert -limit memory 500MiB -limit map 1GiB -size 1920x1088 -depth 16 -define quantum:format=floating-point -define quantum:scale=65536.0 -set colorspace RGB -colorspace RGB rgb:- "%06d_output.tif"

[sneaker_ger]

avs2pipemod supports raw output like avs2yuv too.

You seem to be correct. I didn't got it working in my last tests, but it seems I just picked the wrong mode (stacked instead of interleaved) and wrongly assumed the piping tool would convert the high bitdepth into a different format.

[Yellow_]

hey, thanks that's done the trick. :-)

[SamKook]

I was wondering, would it be possible to pipe the audio and video at the same time? To encode in XviD and MP3 with virtualdub without having to mux them afterwards for example.

[Chikuzen]

update

avs2pipemod-0.1.1.7z(removed)

I rewrote almost code.

* avs2pipemod detects avisynth version, and changes the actions.
　(thus, avs2pipe26mod was removed.)

* avisynth.lib is unnecessary any longer.
　for win64 builds can also be compiled now.

[sneaker_ger]

Thanks.
I noticed a "avs2pipe26mod.exe" is no longer included. Is this obsolete now?

[the_weirdo]

I think the answer is "Yes".


* avs2pipemod detect avisynth version and change actions.
(thus, avs2pipemod26 was removed.)

[sneaker_ger]

I'm blind, thx.

[Chikuzen]

I was wondering, would it be possible to pipe the audio and video at the same time? To encode in XviD and MP3 with virtualdub without having to mux them afterwards for example.

Your request means "I want a realtime streaming encoder only for uncompressed format."

Is it a very difficult subject ?

For present me, yes, it is very difficult...
Although I began to study the C language and one year has passed, it may be able to do, if I will try hard for three or more years.

[SamKook]

I guess I'll forget about it then. It wasn't very important to begin with anyway since I haven't had any crash yet encoding to Xvid straight from avisynth.

I figured that maybe it wouldn't be too hard since it can already do both separately so I might as well ask.

[kemuri-_9]

it is relatively impossible to pipe both streams uncompressed simultaneously, without utilizing a container, to a single stream without the other side knowing exactly what bytes belong to the audio and what bytes belong to the video.

these kinds of scenarios are why container formats exist to multiplex the different streams together into a single stream - the format adds the necessary overhead to indicate which bytes belongs to what stream.

and the amount of effort required here is way beyond what the value added would be.

[Chikuzen]

update

avs2pipemod-0.1.2.7z (http://www.mediafire.com/download.php?47ordinmatla8md)

* fix crash when importing invalid avs.

[b66pak]

thanks a lot...
_

[Chikuzen]

update

avs2pipemod-0.2.1.7z (http://www.mediafire.com/download.php?kbdle1du25hkzqj)

* add new options '-x264raw', '-x264rawtc'.

[Chikuzen]

update

avs2pipemod-0.3.0.7z (http://www.mediafire.com/download.php?ilpfc0m89c95apv)

*add new option '-dumpyuv'

[Aktan]

Thanks for the tool, doobry, and thanks for the modifications, Chikuzen. They work great!

[Bob Wya]

update

avs2pipemod-0.3.0.7z (http://www.mediafire.com/download.php?ilpfc0m89c95apv)

*add new option '-dumpyuv'

Uhmmm not being one to complain - about the great works you are doing - but I believe the GPL v3.0 requires you to distribute the modified source code for this filter as well as the binaries!

Quoting from the license you included with the binaries (only)...
To protect your rights, we need to prevent others from denying you
these rights or asking you to surrender the rights. Therefore, you have
certain responsibilities if you distribute copies of the software, or if
you modify it: responsibilities to respect the freedom of others.

For example, if you distribute copies of such a program, whether
gratis or for a fee, you must pass on to the recipients the same
freedoms that you received. You must make sure that they, too, receive
or can get the source code. And you must show them these terms so they
know their rights.


I only make this pedantic point as I am developing my own Avisynth filter (and I'm poking about in the source code for other filters!) :cool:
Also this filter looks like it would be useful to play about with in Wine (on ARCH-Linux)...

Thanks
Bob

[the_weirdo]

@Bob Wya:
From README.txt:
source code : https://github.com/chikuzen/avs2pipemod

[Bob Wya]

@Bob Wya:
From README.txt:
source code : https://github.com/chikuzen/avs2pipemod

Ahh yes RTFM :readguid:

Apologies to Chikuzen. :D

If I can get it working with ARCH I'll update the WineHQ AppDB page for dependent Windows apps - to use this filter - for a bit of extra publicity! MeGui, for example, is very out of date!! (http://appdb.winehq.org/objectManager.php?sClass=version&iId=16968)

Thanks for helping out an idiot :rolleyes:
Bob

[Chikuzen]

update

avs2pipemod-0.4.0.7z (http://www.mediafire.com/download.php?c6dbv3f2q4pq1w5)
https://github.com/chikuzen/avs2pipemod

* change '-dumpyuv' to '-dumptxt', and support all colorspace.

[Chikuzen]

update

avs2pipemod-0.4.1.7z (http://www.mediafire.com/download.php?33fpx9bibdk8y4j)
https://github.com/chikuzen/avs2pipemod/

* fix wrong error handling in '-x264raw(tc)'

[b66pak]

thanks a lot...
_

[active1]

hi
can i compile it on linux ??

[Chikuzen]

hi
can i compile it on linux ??

avs2pipe/avs2pipemod is a Windows application.
you can compile it with mingw/mingw-w64.
and, I sometimes cross-compile avs2pipemod on Ubuntu.

[Atak_Snajpera]

chikuzen could you add progress indicator similar to bepipes?

[sl1pkn07]

with flag to occult any info (or log to archive). like avs2yuv

[Chikuzen]

Although I had received the request about adding progress indicator from Japanese users several times, they all refused.
The reason is that speed falls by adding it, and it can substitute mostly if the 'frames' option is added to the encoder side.

However, each of these is only the cases of video and is not once requested about case of audio until now.
'similar to bepipes' ... is this case of audio ?

[Atak_Snajpera]

yes i'm looking for something what could replace bepipe for audio only encoding. so far only old bepipe can show progress during processing audio but it requires .net.

[Chikuzen]

avs2pipemod-0.4.2m.7z (http://www.mediafire.com/download.php?alg424t3kx7ak5d)

* add progress indicator for audio output(experimental)

@Atak_Snajpera
Since progress indicators competes when this uses together with some encoders(e.g. QAAC), I'm not going to commit this modified into my repository.
But, I think that it becomes instead of bepipes.

[Atak_Snajpera]

you can always add optional --show-audio-progress switch.

besides most of audio encoders have --quiet switch (opusenc,fhg aac,aften ac3, vorbis,flac)

btw thanks for quick response. i wil test it as soon as i can.

[SamKook]

I'm having a little problem when converting wav that would be over 4GB.
I'm piping audio from an avs script to neroaacencoder, but if I leave the default -wav setting and the audio is too long, it will say the following thing:
avs2pipemod[info]: writing 14504.948 seconds of 44100 Hz, 2 channel audio.
avs2pipemod[warning]: audio size over 32bit limit (4GB), clients may truncate audio.
ERROR: could not parse WAV file
avs2pipemod[info]: finished, wrote 0.011 seconds [0%].
avs2pipemod[info]: total elapsed time is 0.000 sec.
avs2pipemod[error]: only wrote 506 of 639668224 samples.

I know I can fix it by lowering the bitrate, but I'd like to keep the default setting when that's not necessary in my batch files.

Now my problem is that even if avs2pipemod gives an error message, the return value is still 0 meaning there was no errors for DOS so I can't check the %ERRORLEVEL% variable for errors and try again with a lower bitrate. Is there any chance that this could be fixed?

Unless it's me who does something wrong when I test it with the following:
"C:\Program Files (x86)\avs2pipemod-0.4.1\avs2pipemod.exe" -wav test.avs | "C:\Program Files (x86)\NeroAACCodec-1.5.1\neroAacEnc.exe" -q 0.42 -if - -of test.aac & ECHO %ERRORLEVEL%

[tebasuna51]

To allow than NeroAacEnc.exe accept wav files greater than 4 GB you need add the parameter -ignorelength:

...neroAacEnc.exe" -ignorelength -q 0.42 -if - -of test.aac ...

Edit:
BTW the correct extension for output is test.m4a
NeroAacEnc can only output audio AAC in mp4 container.

[SamKook]

Thanks, that make things a lot simpler. I'm not sure how I haven't tried it before since it seem so obvious now that I re-read the help for it.
A different errorlevel in case of errors would still be nice, but much less important now.

Thanks for letting me know about the extension. It doesn't change much since it get muxed right after, but it's always better to do things right.

[_argos]

@Chikuzen (or doobry?, i am confused)
First, let me thank you for your great effort. Nice tool.

I encountered a strange behaviour while trying to capture output of "avs2pipemod.exe -info somescript.avs" in Python. When the script had audio (via DirectShowSource( ... audio=true)) the output could be captured, but without audio (via DirectShowSource( ... audio=false)) thge cpatured output was always empty. Using the command from commandline showed output everytime. So my question ist:

Do you do something to stdout when audio is absent or use different output for info?

Update: Well I looked into the source code. And I don't get it. There is nothing done differently, just an additional if-block for audio, put to sdtout the same way as video. Okay, my new crazy theory: The program exits to fast for Python to capture the output somehow if audio is not present, which then would be a problem of Python not your program. This is weird.

Greetings
Malte

PS I do use -info to probe with different scripts to find out which scripts can be used to render a file.

[feos]

I'm using avs2pipemod to put the video info to file:
avs2pipemod -info encode.avs > info.txt
and right after creating the info file, it freezes. I also did it from the command line without output to file, and it prints the file info in the cmd and then freezes.

Extracting the audio works well. Using the latest version from the above.

[Groucho2004]

I'm using avs2pipemod to put the video info to file:
avs2pipemod -info encode.avs > info.txt
and right after creating the info file, it freezes. I also did it from the command line without output to file, and it prints the file info in the cmd and then freezes.

Extracting the audio works well. Using the latest version from the above.
Post your script and/or try using avsmeter (https://forum.doom9.org/showthread.php?t=165528) (with "-i" switch).

[feos]

Post your script and/or try using avsmeter (https://forum.doom9.org/showthread.php?t=165528) (with "-i" switch).
A note before I do so:
The script alone loads very well into virtualdub, and audio/video processing works from the normal x264 command line. Also, what's notable is that the same batch and the same avs worked together very well for years. Then it stopped working. I installed another OS and it works on neither.

EDIT:

F:\Encode\TASEncodingPackage>AVSMeter encode.avs -i
AVSMeter 1.7.7 (AVS 2.6, x86) by Groucho2004
AviSynth 2.60, build:Sep 18 2013 [17:36:36]
Number of frames: 235326
Length (hhh:mm:ss.ms): 001:05:22.100
Frame width: 320
Frame height: 224
Framerate: 60.000 (60/1)
Interlaced: No
Colorspace: YV12

Then it also froze :/
Here's the script: http://pastebin.com/CNAzMuxj

[Groucho2004]

Also, what's notable is that the same batch and the same avs worked together very well for years. Then it stopped working.
You have to figure out what change causes the script to freeze. Since you mentioned that the script worked for years, you at least changed the avisynth version. It might be a incompatibility between Avisynth and one of the plug-ins.

[feos]

Reverting to older Avisynth version (2010) doesn't help. But It seems the problem may be in using pass=1. The batch sets that before taking info. And when I set pass=1, the script won't load into virtualdub, telling "ExactDedup: failed to open dupinfo file". However the file is where the script reads it, and it looks ok. So the problem is with ExactDedup plugin. But I backupped the dll too, and it doesn't work, while it did (with whatever avisynth version).

[Groucho2004]

"ExactDedup: failed to open dupinfo file".
Try fully qualified paths for the (txt) files.

[feos]

It got loaded into virtualdub at pass=1 now, but avs2pipemod still freezes at the same line.

EDIT: It appears even when I do pass=0 (no ExactDedup used), avs2pipemod still freezes after creating the info file...

EDIT: Using "info" instead of "-info" prints info into cmd window, but creates only the blank file. Which isn't what I need, because that file is read from, to set SAR according to clip parameters.

EDIT: Had to solve it by dropping the FreeSub plugin use. avs2pipemod+freesub+ffmpeg were hanging. Here's some gdb trace: http://pastebin.com/QbPWLqwG.

The issue got its workaround.

[Chikuzen]

new version released

avs2pipemod-1.0.0 (https://github.com/chikuzen/avs2pipemod/releases)
- change avisynth_c.h to Avisynth+MT's avisynth.h.
- change C99 to C++11.
- change mingw-w64 to VisualC++2015.
- drop Avisynth2.5x support.
- drop WindowsXP support.
- drop x264bd and x264raw options.
- add Avisynth+MT's new 16/32bit formats support.
- add FFmpeg's extended Y4M formats output support.
- add '-dll' option.

now, avs2pipemod requires VC++2015 runtime (because I decided to avoid avisynth_c.h).

[Chikuzen]

updated.

avs2pipemod-1.0.1 (https://github.com/chikuzen/avs2pipemod/releases)
- fix video output when avs has no audio (reported by stax76).

[Chikuzen]

updated.

avs2pipemod-1.0.2 (https://github.com/chikuzen/avs2pipemod/releases)
- fix calculating video duration of -info.
- show avisynth's VersionString() on -info.

[Chikuzen]

updated.

avs2pipemod-1.0.3 (https://github.com/chikuzen/avs2pipemod/releases)
- fix memory leak possibility.
- add version resource.

[ultim]

updated.

avs2pipemod-1.0.3 (https://github.com/chikuzen/avs2pipemod/releases)
- fix memory leak possibility.
- add version resource.

Just being curious: Did you find the memory leak by chance, or were you made aware of it by Avs+'s new warning logs?

[Chikuzen]

Just being curious: Did you find the memory leak by chance, or were you made aware of it by Avs+'s new warning logs?
by Avs+'s warning.

[yup]

Hi all!
Please advice how use avs2pipe with x265.
I can write command for rawvideo:
avs2pipemod64 -rawvideo test.avs | x265_64 --input - --input-res 960x720 --fps 50.0 --output video.h265
How can write for y4m, script output progressive video.
yup.

[Chikuzen]

avs2pipemod64 -y4mp test.avs | x265_64 --input - --y4m --output video.h265

[yup]

Chikuzen!
:thanks:
yup.

[leoenc]

Is it possible to use both a pipe and a standard input?

I'm trying to pipe the video to ffmbc and specifying audio input for the ffmbc side:

avs2pipemod.exe -y4mp video_source.avs | ffmbc_64.exe -i - -y -i audio_source.avs -vcodec prores aspect 16:9 -threads 8 -profile hq -acodec pcm_s16le output.mov -acodec pcm_s16le -newaudio -map_audio_channel 1:1:0:0:1:0 -map_audio_channel 1:1:1:0:2:0

But it fails with the following message:

audio_source.avs: Invalid data found when processing input

[leoenc]

Ok, the error I got above is because I was trying to load 32-bit AVS into 64-bit ffmbc.
Is there a way to load multiple avs files?
E.g.:
avs2pipemod.exe -y4mp video.avs audio.avs | ffmbc_64.exe -i - -y -vcodec prores aspect 16:9 -threads 8 -profile hq -acodec pcm_s16le output.mov -acodec pcm_s16le -newaudio -map_audio_channel 1:1:0:0:1:0 -map_audio_channel 1:1:1:0:2:0

[Aktan]

Ok, the error I got above is because I was trying to load 32-bit AVS into 64-bit ffmbc.
Is there a way to load multiple avs files?
E.g.:
avs2pipemod.exe -y4mp video.avs audio.avs | ffmbc_64.exe -i - -y -vcodec prores aspect 16:9 -threads 8 -profile hq -acodec pcm_s16le output.mov -acodec pcm_s16le -newaudio -map_audio_channel 1:1:0:0:1:0 -map_audio_channel 1:1:1:0:2:0

Can't you just combine the video and audio in one AVS? The function is AudioDub().

[leoenc]

I'm afraid not as it's being used in some automation scripts which expect separate inputs for video and audio.

[konstantin1]

When I try to use this utility I get only an error message: "avs2pipemod[error]: failed to create avisynth script environment."
Till now I used the makeavis.exe utility from an ffmpeg version. But I often got zero division error in Wine.
In fact I can't install avisynth, because I use it on a headless virtual server. So I usually copy the avisynth.dll and devil.dll files to the actual directory, and make programs pick them up from there. This scenario worked with makeavis.exe and avs2yuv.exe too.

When I use the option -dll I get an error message again:
wine avs2pipemod -wav=24bit -dll=Z:\opt\lampp\htdocs\thumb9\hfaek\avisynth.dll p2.avs > 1.wav

avs2pipemod[error]: failed to load avisynth.dll

My avisynth.dll is from a 2.6 version install.
What can I do in this special case to make the utility work?

[shae]

Both avs2pipe 0.3 and avs2pipemod 1.1 get stuck at the end of encoding, using AviSynth+ and fdkaac.
Any idea why?

avs2pipe26_vs audio 1.avs | fdkaac.exe -m 5 -p 2 - -o 1.m4a
or
avs2pipemod -wav 1.avs | fdkaac.exe -m 5 -p 2 - -o 1.m4a

[tebasuna51]

Without problems here with avs2pipemod 1.1.1, Avs+ r2508, fdkaac 0.6.2 and W 7 SP1.

Please show your 1.avs

[shae]

I didn't think a script that works fine could be the cause, but you're right.

It gets stuck because of QTGMC.

[tebasuna51]

Any video management in the script is usseless (and time consuming) for audio conversion.

You must use a simplified script taking care only if video duration is changed in the script, for instance:

OpenAudioVideoSource("movie")
Trim(x,y) # or other functions than change duration
KillVideo() # the video is ignored after that
...Only audio functions...

[shae]

Sure it's doesn't matter to audio, but having separate scripts for audio and video is messier and more error-prone. The video processing parts don't seem to affect speed when only the audio is used. At least, I haven't seen any difference.

The mystery is why it gets stuck due to QTGMC.

[poisondeathray]

What about using ffmpeg instead ? You can compile with libfdkaac and encode video & audio & mux at same time.

[shae]

It's not that big a deal, just a bit uncomfortable. For a single encode I just force-close the console window. For multiple I comment/uncomment a few script lines before/after audio encode.

I don't like ffmpeg for encoding when quality matters. It's always in flux and you never know what specific versions of sub-component were compiled into it, or what are the implications.

(Off topic: ) BTW, I switched the AAC encoder from FDK to FhG (DLLs from Winamp, fhgaacenc for CLI interface). It's 3-4 times faster. I don't know if there's a quality difference, but if there is, I'm guessing FhG might be better because it's from a desktop software, while FDK is based on some Android code.

[Stereodude]

There's no way to control the affinity of the x265 encoder spawned by avs2pipemod from the command line. If you launch avs2pipemod using the start command with affinity restrictions it only affects avs2pipemod, not the spawned x265 session.

I couldn't find any way to feed the output of a .avs to x265 with affinity restrictions. Is there another piping tool or utility that can do it?

[Yanak]

I don't use avs2pipemod in command line and don't know the command lines you use for this but maybe this i posted here can help :
http://forum.doom9.org/showthread.php?p=1834344#post1834344

I doubt you can call a .bat file directly from avs2pipemod but that could also be a way to do it , "start /affinity 0xfc BatchFile.bat" or simply the batch file having the command in the link above + your x265 commands.

Not sure if this can help but who knows...

[Stereodude]

I don't use avs2pipemod in command line and don't know the command lines you use for this but maybe this i posted here can help :
http://forum.doom9.org/showthread.php?p=1834344#post1834344

I doubt you can call a .bat file directly from avs2pipemod but that could also be a way to do it , "start /affinity 0xfc BatchFile.bat" or simply the batch file having the command in the link above + your x265 commands.

Not sure if this can help but who knows...
Thanks, but this is what I was saying doesn't work. The start command with the affinity and NUMA nodes switches only affects avs2pipemod in the .bat file, not the x265 encoding session it spawns/calls.

[Yanak]

Arg, sorry then, I had in idea ( and not sure if it was possible or not ) the av2s command or bat file calling another batch file where the x265 part only is present and having the affinity set there but if this doesn't work i don't know sadly :/

[Aktan]

Assuming you are running only 1 x265 at a time, you could find x265 and then set the priority via command:

https://stackoverflow.com/questions/23996307/set-process-priority-on-already-running-process-batch-file

Edit: I only did a quick look, but I'm sure there are more tricks like getting the PID instead to find the right x265 if you need to change the priority of multiple x265s.

[Stereodude]

Assuming you are running only 1 x265 at a time, you could find x265 and then set the priority via command:

https://stackoverflow.com/questions/23996307/set-process-priority-on-already-running-process-batch-file

Edit: I only did a quick look, but I'm sure there are more tricks like getting the PID instead to find the right x265 if you need to change the priority of multiple x265s.
I know it can be changed after starting, but x265 spawns more threads to start with because it sees all the logical processors in the system. Restricting it after it starts doesn't fix that. If you start a program correctly with restricted affinity the program only detects those cores allocated to it and spawns less threads as a result. I did this with x264 regularly. I also do it with handbrake.

[qyot27]

The most important question here is 'why are you using avs2pipemod in the first place'? The answer to that will change the solution.

If you're using it simply as a piping tool to get y4m from AviSynth to x265, but both are still 32-bit (or 64-bit), then the solution is to use either libx265 through FFmpeg, or the x265-Yuuki patchset from msg7086 (assuming the build of x265-Yuuki has its LAVF input enabled, and was built against an FFmpeg configured with --enable-avisynth). In those cases, there's no piping, so the affinity should take without any issue.

If you're using it to get 32-bit AviSynth output to input 64-bit x265, then you could try to manually pipe, but insert the affinity command after the pipe, at the beginning of the x265 command. That is, if it allows you to do so.

[Aktan]

I know it can be changed after starting, but x265 spawns more threads to start with because it sees all the logical processors in the system. Restricting it after it starts doesn't fix that. If you start a program correctly with restricted affinity the program only detects those cores allocated to it and spawns less threads as a result. I did this with x264 regularly. I also do it with handbrake.

If it's anything like x264, you can just set the threads param yourself...

[Asmodian]

Good point, does "--pools" or "--numa-pools" not work? It doesn't seem to let you restrict to specific cores, only specific NUMA nodes.

[Stereodude]

The most important question here is 'why are you using avs2pipemod in the first place'? The answer to that will change the solution.

If you're using it simply as a piping tool to get y4m from AviSynth to x265, but both are still 32-bit (or 64-bit), then the solution is to use either libx265 through FFmpeg, or the x265-Yuuki patchset from msg7086 (assuming the build of x265-Yuuki has its LAVF input enabled, and was built against an FFmpeg configured with --enable-avisynth). In those cases, there's no piping, so the affinity should take without any issue.

If you're using it to get 32-bit AviSynth output to input 64-bit x265, then you could try to manually pipe, but insert the affinity command after the pipe, at the beginning of the x265 command. That is, if it allows you to do so.
I'm using it to get the avisynth output into x265 since it can't accept an .avs natively. I'm not crossing 32-bit to 64-bit or anything like that.

Good point, does "--pools" or "--numa-pools" not work? It doesn't seem to let you restrict to specific cores, only specific NUMA nodes.
I didn't try them. I wanted to avoid having to handle all that manually.

[Kurt.noise]

For those who interested, I updated avs2pipemod in order to include channelmask information from latest avisynth+.

https://github.com/Kurtnoise-zeus/avs2pipemod/releases

[tebasuna51]

Thanks Kurt, I'll test it.

[EDIT]Works fine for me thanks

[Chikuzen]

Hi.

version 1.3.0 is released.
https://github.com/chikuzen/avs2pipemod/releases/tag/1.3.0

- UTF8 filename support
- AVS+ internal ChannelMask support
- new option 'dumpprops'

dumpprops outputs frame properties as JSON.

From now on, only the 64-bit version will be distributed.

Bye.

[real.finder]

Hi.

version 1.3.0 is released.
https://github.com/chikuzen/avs2pipemod/releases/tag/1.3.0

- UTF8 filename support
- AVS+ internal ChannelMask support
- new option 'dumpprops'

dumpprops outputs frame properties as JSON.

From now on, only the 64-bit version will be distributed.

Bye.

nice to see you back!

thanks for the update! BTW, did you noted that some plugins you made get some update from https://github.com/Asd-g?tab=repositories ? like https://github.com/Asd-g/RawSource_2.6x and https://github.com/Asd-g/DCTFilter and https://github.com/Asd-g/yadifmod2 and https://github.com/Asd-g/ReduceFlicker/

[Chikuzen]

nice to see you back!

thanks for the update! BTW, did you noted that some plugins you made get some update from https://github.com/Asd-g?tab=repositories ? like https://github.com/Asd-g/RawSource_2.6x and https://github.com/Asd-g/DCTFilter and https://github.com/Asd-g/yadifmod2 and https://github.com/Asd-g/ReduceFlicker/

I know that other people have made various changes to them, but I haven't seen it all.
will use them as a reference for my future work, but if I want to make any changes, I will probably do it myself.

Other than this, I made some changes to RawSourcePlus.
https://github.com/chikuzen/RawSource_2.6x/releases

It's hard to remember what I did almost 10 years ago.

[Chikuzen]

I found a bug and updated.

1.3.1
https://github.com/chikuzen/avs2pipemod/releases/tag/1.3.1

[FranceBB]

Nice to see you back Chikuzen, it's been a while since 2016! :)
Thanks for the updated version.