1 avs output to 2 VDubs same time?

[Zep]

is there a way to get 1 script to send that data out to 2 instances of VDub
(insert you fav app here) so that the decoding of the mpeg2, decimate, filters etc....
need only be done once,

Now for the why i want this. The above takes a lot of time and doing it twice,
once for each pass is a wasteful. So if you could do pass 1 while at the same
time saving the output to as an example mjpeg set to max quality which is very
fast then you could use that output as the input source on pass 2. You wouldn't
even need to use avisynth even for pass 2 and could get rid of even more overhead
by loading it directly into VDub.


Of course it would be cool if avisynth could encode directly. Example

AviEncode("c:whatever.avi", singlepasscodec=mjpeg) # huffy or VBLE etc...



thoughts?

[ObiKenobi]

Originally posted by Zep
Now for the why i want this. The above takes a lot of time and doing it twice,
once for each pass is a wasteful. So if you could do pass 1 while at the same
time saving the output to as an example mjpeg set to max quality which is very
fast then you could use that output as the input source on pass 2. You wouldn't
even need to use avisynth even for pass 2 and could get rid of even more overhead
by loading it directly into VDub.

Why don't you just encode the filtered output to a lossless format then encode that file with whatever other codec you want.

Originally posted by Zep
Of course it would be cool if avisynth could encode directly. Example

AviEncode("c:whatever.avi", singlepasscodec=mjpeg) # huffy or VBLE etc...

Why? There are plenty of other encoding applications out there to use that can already do exactly what you want. Spending time to develop an internal encoder would just be a waste of time.

[Zep]

Originally posted by ObiKenobi
Why don't you just encode the filtered output to a lossless format then encode that file with whatever other codec you want. Why? There are plenty of other encoding applications out there to use that can already do exactly what you want.

because that is 3 passes and you lose a ton of time on 3 passes
which is exactly what I want to avoid. I hope you realize that mjpeg
uses only 10% on a decent CPU so you could do Xvid pass 1 at the
same time and since mjpeg (or a lossless codec) make large files you get a
bottleneck there and another reason to read/write to/from a 10 gig file
as little as possible.

your way means writing out a huge 10 gig file and letting 90% CPU go idle
during then read that huge file back in *2* more times for each pass in Xvid so
that is 3 passes really.

my way means you waste no CPU because while you write the file you
use the data in memory **right then and there** to do pass 1 and you should
since the data IS there. This means you only read the 10 gig file
on pass 2 which is 1 less than your way.

There is just too much overhead in 3 passes and it nullifies the speed gain
you could get.

[Leak]

Originally posted by Zep
because that is 3 passes and you lose a ton of time on 3 passes
which is exactly what I want to avoid. I hope you realize that mjpeg
uses only 10% on a decent CPU so you could do Xvid pass 1 at the
same time and since mjpeg (or a lossless codec) make large files you get a bottleneck there and another reason to read/write to/from a 10 gig file as little as possible.

Ah, but if MJPEG compression only takes 10% CPU time and your filter combo is very CPU intensive then AviSynth will use the other 90% so no CPU cycles will go to waste.

The other 2 passes will then be much faster since it's mostly direct MJPEG to XviD compression, so doing 3 passes in all will hardly be slower than 2.

The only way you'd get a big speed difference would be if your filtering took little CPU time, and then you could just do the filtering on each pass anyway.

[Zep]

Originally posted by Leak
Ah, but if MJPEG compression only takes 10% CPU time and your filter combo is very CPU intensive then AviSynth will use the other 90% so no CPU cycles will go to waste.

The other 2 passes will then be much faster since it's mostly direct MJPEG to XviD compression, so doing 3 passes in all will hardly be slower than 2.

The only way you'd get a big speed difference would be if your filtering took little CPU time, and then you could just do the filtering on each pass anyway.

Not in my tests. 2 passes are 40% faster than 3 passes
here on my PC's.

You can do a simple test yourself using just Xvid even.
note how reading in a 10 gig file is a bottleneck.

First lets do it your way.

We read in the 10 gig .ts file, decode it, decimate and scale

The chain, Avisynth ---> VDub ---> Xvid
we set to single pass Q=1. no b-frames we want max quality. all Q=1
frames writen to that 10 gig file. Now we have our new source file
decimated, and scaled and whatever all done to it.

VDub pass 1 we read that 10 gig new source file in
VDub pass 2 we read that 10 gig new source file in *AGAIN*

3 passes and 3 reads of bottleneck 10 gig files.
(1 ts read, 1 avi write, 2 avi reads)



Now my way

We read in the 10 gig .ts file, decode it, decimate and scale.

The chain, Avisynth ---> VDub ---> Xvid
pass 1 ZONE ALL set to Q=1 and disgard pass 1 file turned off.
We want the avi and the over head is about 15% (one reason for mjpeg
since the over head is much lower. the other reasons in a minute)

VDub pass 2 we read that 10 gig new source file in

just 2 passes and just 2 reads of bottleneck 10 gig files
(1 ts read, 1 avi write, 1 avi read)


my test show 40% time savings when all is said and done and that
does NOT include the extra set up time and babysitting to have
to do to set up an extra pass with your way.


now you may be wondering if that works so well why do i want mjpeg?
Well for one thing it encodes super fast as well as decodes super fast.
next, every frame is a key frame. Editing/scanning a mjpeg file is
super fast and you have to cut on a key/i frame. yes now and then
i do go back and need more editing and it is much better to do it
on a mjpeg than the xvid file with p frames. (though you could
make them all i frames i guess but then the file is super huge)
some times I want to cut out MORE, like credits, if the final encode
is not as good as i want and i want teh same target file size)

and now the most important reason. Setting pass 1 zone all to
Q=1 throws off the xvid calcs a little. i.e. the video.pass file
is not as accurate and you then need to change the target size
some to get the size you really want. syskin and i talked
about this already. it is a minor draw back but one none
the less.

Now if you had a 200 meg a second RAID, the file read/write
would not matter as much. me i do not, so i read the ts
from 1 hard drive and write the output.avi to another hard drive
which helps some but it is still a bottleneck because
reading in that much data does take a good bit of time.
in mjpeg's case it can decode much faster than the drive can
feed data and thus why so much CPU is idle. The decoder/encoder
just can't be fed fast enough. typical modern day PC i/o bottleneck
is the hard drive and why RAIDS are becoming so popular :D

[stephanV]

the bottle neck is hardly the reading speed from your HD, the bottle neck is the CPU the filtering takes up. if your HD IS the bottleneck, buy a new one.

i even dare to say that encoding from a uncompressed source is faster than from any compressed source (assuming no filtering or the same amount of filtering is used and no extraordinary resolutions are used)

furthermore, MJPEG is not lossless...