how does n-pass-encoding work?

[NetSoerfer]

Uhm...

maybe this is a stupid question, but am I the only one not to know how to encode with more than 2 passes? I did a search in this forum, but didn't find anything that would have helped me...

What do I have to do to make more passes?


:confused: soerfer :confused:

[hakko504]

After having saved the avi once with the settings you want for the second pass, just re-save the avi with the same settings. (assuming you are using the job-list in VD)

[symonjfox]

I tested it quite a lot, but I'm quite perplessed.

The first pass analizes, ok?
The second, reads the log file and produce a Vbr file, ok?

The other passes, what do they do?
Do they re-analize / re-encode?
How do they re-allocate the bitrate if they rewrite the previous pass file?
Do they update the log file while encoding?
If they do this, why don't they create a "Last nth pass" mode so in the last pass the codec won't continue updating the log file (which will be useless).?

I tell you this because I didn't see so much quality improvement between passes (maybe I need a pair of glasses :p ).
I guess if I have to spend 5x time to encode a movie, while before I have always used Quantizer mode (with Q2 or 3 for long movies).

[NetSoerfer]

so, just put the same job several times into VDub's job list? I tried that, the files were EXACTLY the same size... or do I have to change the filename?

by the way, I keep getting an question message at the beginning of the nth pass whether I want to overwrite the existing log - can you make anything of that?


soerfer

[symonjfox]

there's an option that "Do not prompt with errors and messages", just enable it (I guess why they didn't enabled it by default).

[hakko504]

>The first pass analizes, ok?
Yes
>The second, reads the log file and produce a Vbr file, ok?
Yes

>The other passes, what do they do?
>Do they re-analize / re-encode?
Yes, given the results of the previous passes.
>How do they re-allocate the bitrate if they rewrite the previous pass file?
The .avi generated from the previous pass is obsolete when you do more than two passes.
I think it is a matter of redistributing the leftovers of the bitrate reservoir from the previous pass.

>Do they update the log file while encoding?
Yes, if you check the box 'update log'
>If they do this, why don't they create a "Last nth pass" mode so in the last pass the codec won't continue updating the log file (which will be useless).?
Just uncheck the box marked 'Update log' if you are absolutely sure you won't re-encode the video again.
>I tell you this because I didn't see so much quality improvement between passes (maybe I need a pair of glasses :p ). I guess if I have to spend 5x time to encode a movie, while before I have always used Quantizer mode (with Q2 or 3 for long movies).
Actually, a correctly created 2nd pass shouldn't be possible to improve on. DivX5.02 showed that their VBR encoding routines are far from perfect. And if you are used to doing Q2 or Q3 rips there is no reason to start doing VBR just because they are multipass.

@soerfer
Strange, I'll have to check that. I haven't really done any tests yet with the multipass feature, I just set everything up yesterday, and started the encodes before I left home this morning.

[temporance]

Originally posted by hakko504
Actually, a correctly created 2nd pass shouldn't be possible to improve on.Not true. When an encoder encodes the first pass (or a single pass) it is making lots of guesses of how to set the quantizer etc. for each frame in order to achieve the required bitrate. The second time through, the encoder is still guessing what will happen but this time it has more knowledge to make an accurate guess and come close to the target bitrate. But two passes is not enough to get fair distribution of bits between frames. After the second pass, the encoder has a very good idea of the optimum way to encode each video frame.

So if we think in terms of % possible quality, a single pass encode might hit 60%, a 2-pass encode might get near 90% and a 3-pass encode would be around 98%. Doing five or ten passes is probably a waste of time.

And if you are used to doing Q2 or Q3 rips there is no reason to start doing VBR just because they are multipass.True. Unless you want to use the modulation to devote more bits to low motion scenes.

[hakko504]

@temorance

You just pointed out why DivX5.0x doesn't create perfect two-pass encodes, they make a lot of guessing in the first pass. If the first pass would collect all possible data about the clip then the second pass would instantly know how to set up the encode in the best posible way and from this it would be very hard to improve, no matter how many passes you do.

Take XviD or nandub (DivX3.11) for instance. They set the quantizer to 2 in the whole first pass, thus collecting as much data as possible, and can then use it to make the best possible choises already for the second pass. They do not guess what the settings should be, but analyzes the whole clip without any prejudices.

[temporance]

Running a single pass with q=2 does not obtain every single bit of information about a clip. In the second pass, xvid still needs to guess which quantizer to use for each frame, especially when that quantiser is not close to 2.

To simplify greatly, VBR rate control has one aim - to discover a constant quantizer that, when applied to every frame, will hit the desired filesize. Xvid does not achieve a constant quantiser for the 2nd pass, it varies a lot. I haven't tried 5.0.3 Pro yet.

If you have the compressed frame sizes for a whole video encoded at q=2 (which produced, say, 1645kbit/s), how would you set the quantizer in order to achieve 710kbit/s? When you ask yourself this question you realise that encoding the clip with q=2 does not give you enough information. There is an approximation, compressed_frame_size = f(q), but even this leaves you guessing and will result in varying quantizer during the 2nd pass.

Like I said, I would guess that 5.0.3 would give this:

single pass: 60% potential quality
2-pass: 90% potential quality
3-pass: 98% potential quality
4-pass: 99% potential quality
5 pass: 99.5% potential quality

And that two passes with DivX nth pass encoding would be at least as good as old 2-pass methods. Only DivX gives you the chance to try more passes, if you want.

[tripnotik]

Originally posted by NetSoerfer
so, just put the same job several times into VDub's job list? I tried that, the files were EXACTLY the same size... or do I have to change the filename?
soerfer


They may be the same size but their content is different, just do a binary compare of the files and you'll see. But after many passes, they will always be the same. For example, for a test clip I experimented with, the 6th and 7th pass were identical (except 4 bytes), meaning there is no need to do more passes.

[TheXung]

If you think XviD and nandub do a perfect 2-pass then you are highly mistaken. Each frame has it's own rate-distortion curve, i.e. different frames shrink by different percentages as you increase quantizers due to the nature of the frame and the distortion of it's reference frame. XviD and nandub haven't the slightest clue about the rate-distortion curve. I think they're just assuming that's a certain curve shape and making judgements based on that. Think of multi-pass as testing different points of rate-distortion and eventually coming to an optimal one for each frame.

[iwod]

So to my understanding, Npass is just number of pass, and with each pass it actuallly create a REAL Video AVI file, and every pass after that would overwiting the AVI file in the previous pass until the N number is reached.

But if it is only analysing the file with each pass it is ganing more and more knowledge, why does it have to write a Real AVI file?? can it just record down the result, more like simulate the process until the last pass to actually encode the files with the results??

[symonjfox]

Originally posted by tripnotik
But after many passes, they will always be the same. For example, for a test clip I experimented with, the 6th and 7th pass were identical (except 4 bytes), meaning there is no need to do more passes.

Another nice option to do to DivX 5.04 :D The program should be able to decide if another pass would increase quality, or not (and the user should tell the program a percentage of tollerance ... for example if increase is 0.001% there's no need to reencode the entire movie another time, don't you think so?)

So, why don't they create a STANDALONE ENCODER? For example, the most users just use AVISYNTH to adjust their videos and Virtual Dub is used just to FAST RECOMPRESS and multiplex audio (with many limitations example no VBR audio or no AC3). Using a Standalone encoder, there's no need to use Vfw codec and it's possible to automatize many things. For example it would be possible to binary compare the files ...

I studied a little bit of programming at school, but I wouldn't be able to write a program like this ... maybe someone else will do it :rolleyes:

Originally posted by iwod
But if it is only analysing the file with each pass it is ganing more and more knowledge, why does it have to write a Real AVI file?? can it just record down the result, more like simulate the process until the last pass to actually encode the files with the results??

Good question! Another point ... the standalone encoder should do it without writing another avi file, it would just catch its own statistics in a *.dat file where it can write down everything it wants ... then, in the last pass, just write down the real AVI (or MP4 if they like it :sly: )

[Rash]

@iwod

It writes a real AVI simply because it doesn't know how many passes you want it to do.

So let's say you did 3 passes and saw the movie. Maybe you think that the movie could be improved by doing 2 more passes. Then just open your software and do 2 more passes.

That's it, that's why it creates a valid AVI all the time from the 2nd pass and it keeps updating the .log file (even if you won't encode anymore).

Besides, writing the AVI or not won't improve encoding speed anyway. ;)

[angelyote]

also keep in mind that lower bitrates are going to benefit the most from multi-pass encoding. If you're ripping at a high bitrate (low average quantizer in analyse.log) then you probably aren't going to see much improvement doing 5 passes or whatever.

dave

[iwod]

Originally posted by Rash
@iwod

It writes a real AVI simply because it doesn't know how many passes you want it to do.

So let's say you did 3 passes and saw the movie. Maybe you think that the movie could be improved by doing 2 more passes. Then just open your software and do 2 more passes.

That's it, that's why it creates a valid AVI all the time from the 2nd pass and it keeps updating the .log file (even if you won't encode anymore).

Besides, writing the AVI or not won't improve encoding speed anyway. ;)

I don't quite get it...

so you are saying i said to the encoder to do three pass. then the encoder will do three pass and have the final version...

Now after i saw the final AVI i think it need more pass. So i will add the file to the encoder to do more pass. as you said 2 more pass....

Then the encoder MUST know that it has to stop at 3 pass in the first time round. I know it may not speed it the process since it is like first just takes as long as secound pass. But why do something that to me seems totally unnessacery??...............

[LeonMcNichol]

It's Nth passes, which means, you can do more than 2 passes. It creates an avi, so you can just stop at the pass you want. Now, if they added a feature that had Nth first passes, that would be nice. Maybe instead of calling them 1st pass and 2nd pass, call it Analyze and Write. That way you can just do the analyze pass over and over again.

The nice thing about the Nth pass method, you can compair the differences between the passes. (If you opt to rename them, instead of write over.) This way you can tell what would be a good idea to use. I found 5 is about the max you really need to go. Anymore, you are just wasting time. There is a significant difference between pass 2 and pass 5. (I'll probably be doing pass 2 though, I need a faster computer. =/ )

[dTb]

@iwod
You never tell the encoder how many passes you want. In vdub for instance you set up the multipass - first pass and add to job list. Then for the second pass you set up multipass - nth pass and add to job list. Now, for each extra pass you do exactly the same thing as you've done for the second pass. The codec never has any clue as to how many passes you will do so it writes the avi after each multipass - nth pass.

[OUTPinged_]

People, wait a minute.

All that Npass thing is just a way to distribute overflow more better, right?

So the less overflow codec produces, the less need for one more pass.

After all, it comes down to chosing a quantizer, ne?


Now let's assume this: i am hitting filesize well enought with 2pass. I cap min and max quantizer and everywhere quants are chosen just about right. And I dont care if frame X will get quant4 or 5, i just need extremes to be handled ok. Will I benefit from Npass? ;-)

[Acaila]

All that Npass thing is just a way to distribute overflow more better, right?No, nth pass is for better bitrate distribution. Overflow might benefit from it, but that's not the primary reason for its existence.

And I dont care if frame X will get quant4 or 5, i just need extremes to be handled ok. Will I benefit from Npass? ;-)No.

Nth-pas encoding is only to "get the most out of your bitrate". It's for optimizing quality to the extreme. If you're satisfied with the almost-perfect distribution of the normal 2-pass method, then there's no need to spend more passes to get a near-perfect distribution.