Multi pass or Single pass in AutoGK

[Disco]

There is only one thing I miss in AutoGK and that is the possibility to set if it should render in multi pass or just single pass.

Or is that possible?

The only way I've so far been able to make it render multi pass is to set a fixed file size output. Otherwise it seems to choose single pass.

Or am I doing something wrong?

[len0x]

If you know exact target size then two-pass encoding is used.
If you want a given quality without given size then single pass is used.
Why do you need anything else ?

[Disco]

Well, isnt Multipass always better? Better quality?
So what Quality setting should I use to get it to render in multipass?

As I understand it singel pass files are bigger and looks less good than a mulitpass rendered movie.

[manono]

Hi-

I'm not sure you really understand what you're asking. Some of it is hotly discussed, and I'll just give you my opinions.

Well, isnt Multipass always better? Better quality?

No. The advantage of using 2 or more passes is that you can hit your desired file size accurately (as len0x stated above). So if you're going for a 1 or 2 CD rip, then 2 (or more) passes is the way to go. My opinion about a quality encode is that it always produces better results for a similar file size, when compared to a 2-pass encode. Why? Because, unless you limit your quant ranges severely (as is possible in XviD), the quality encode will give you a more even quality throughout the entire movie, because only 2 quants are used. A 2-pass encode will ordinarily use a much greater range of quants, and those frames/scenes using the higher quants will look worse than others. The scenes with the higher quants will ordinarily be the complex or action scenes, and some might argue that it's better to have them at a somewhat lower quality, to help improve the more static scenes where the quality differences are more obvious. I just don't happen to agree with that. It's a personal preference.

As I understand it singel pass files are bigger and looks less good than a mulitpass rendered movie.

Maybe in the old days, when single pass meant CBR encoding, that was true. It's certainly not true with a quality encode.

So what Quality setting should I use to get it to render in multipass?

You use a 1-pass quality setting (the default 75% is good) if you don't much care about file size. Or you use a 2-pass encode for 1 or 2 or 3 CDs. It's one or the other but not both. By the way, a 1-pass 75% quality encode will turn out roughly similar to a 2-pass encode at the same resolution, where the compression test result is also around 75%. Maybe that's what you meant to ask. I'm not real sure.

[Disco]

Thank you very much for sorting that out. :)

[electrongunner]

You use a 1-pass quality setting (the default 75% is good) if you don't much care about file size. Or you use a 2-pass encode for 1 or 2 or 3 CDs. It's one or the other but not both. By the way, a 1-pass 75% quality encode will turn out roughly similar to a 2-pass encode at the same resolution, where the compression test result is also around 75%. Maybe that's what you meant to ask. I'm not real sure.

I'm afraid you are confusing people into thinking there's no reason to use a 2-pass encode when in reality there are still very compelling reasons to use a 2-pass encode. I've been having this discussion with someone over at the Snapstream forums who is trying to use this post as proof that there's no reason to use 2-pass encoding. I think you need to make sure people know that while they CAN get excellent quality in shorter time when encoding 1-pass, the resulting files will ALWAYS be MUCH larger to achive the same results you'd get from a 2-pass version. In my case, I've got input files that are so noisy that a 1-pass encode at 75% results in files that are actually LARGER than the original input files. The only way to get the best quality at the smallest possible file size is via a 2-pass encode. I can generally reduce files to 25% of their original size using 2-pass encoding, whereas to get the same quality out of a 1-pass encode I end up with HUGE files and the more noisy the source, the more bloated the output files become in 1-pass mode. They also bloat a little in 2-pass as well, but not nearly as much as in 1-pass mode. For people who want the best quality at the smallest size, it's still a 2-pass world.

[electrongunner]

Originally posted by len0x
If you know exact target size then two-pass encoding is used.
If you want a given quality without given size then single pass is used.
Why do you need anything else ?

I think people are looking for a way to do a 2-pass encode that will be a specific percentage smaller than their original input files. There's a bunch of PVR software users that are scripting AutoGK to automatically compress their recorded TV into MPEG4 and who want to be able to automaticaly specify a size realative to the input files (don't worry it's nothing illegal). So they are trying to specify a percent for AutoGK, but they are getting a different percent than they think they're getting. I've managed to hack the script they were using, so now they can send AutoGK specific file sizes that are calculated as a percentage of the input file size. It would be cool if you made your percentage encoding so that it could work both ways, so people could specify a percent and either get a 1-pass quality based on the percentage, or they could get an output file that is X% the size of the input file.

[jggimi]

A 1-pass percentage is based on the Quantizer used, not the file size. Such an encoding is not to any set size.

To obtain a specific file size, based on a percentage of input, CDs, or any other arbitrary size, a 2-pass bitrate based encoding is required.

For more information on 1-pass, 2-pass (or N-pass), Quantizers, bitrate based encoding, quality based (Quantizer based) encoding: see the DivX User Guide (http://www.divx.com/divx/divxpro/guides/). These encoding attributes are valid for both DivX and XviD.

[manono]

Hi-

...the resulting files will ALWAYS be MUCH larger to achive the same results you'd get from a 2-pass version.

Nonsense, not if all other variables are the same, including the resolution, ave. quant, resizer, matrix used, and the audio bitrate. If you doubt that, then do an encode both ways, and then compare the results in GSpot and in DRFAnalyzer (and take note of the log that tells you what changes were made when compared to the 1-pass mode). It is difficult to tell what matrix and resizer is used (except for the AutoGK log), but switching the matrix and/or resizer for XviD encodes can help the compression a lot.

I repeat, the only reason to use 2-pass encoding is to reach a desired file size (like for 1 or 2 CDs, or 1/2 or 1/3 of a DVD-R, or even for 1/4 the original file size). It's nearly impossible to tell what final size AutoGK is going to produce before doing a 1-pass quality based encode. With experience, you can learn to judge the compressibility of the source, and make an educated guess. Or you can do a compress test in GKnot first to get an idea of its compressibility.

And as jggimi mentioned, the percentage figures don't correlate well to final file size compared to original source file size. To get half the original file size, the percentage should probably be 40% or so (all other variables remaining the same). I think you've done some studies on that subject. But even that depends on the compressibility of the source.

The 2-pass encode might be slightly smaller, but mostly because the quants aren't as tightly restricted as for a 1-pass quality encode. Or the resizer/resolution/matrix were changed. That said, I do use 2-pass encoding for my own encodes, but for the 2nd pass (in VDubMod, manually) I also restrict the quants just as for a quality based encode. Once you learn how to do it, the file size always comes out OK.

It is true that noisy sources are much more difficult to compress well, and because the grain/noise covers up a lot of the problems, a 50% quality based encode may look nearly as good as a 75% encode, and if you've read other posts of mine on the subject, you'll see where I recommend that for hard to compress sources (that and/or lowering the resolution). It's also true that MPEG-4 doesn't handle grain and noise very well, and it's indeed possible to wind up with a result even larger than the original. It's even possible with DVD sources. I've seen it many times.

[electrongunner]

Okay, I admit it, I had my head up my ass. I based many of my initial conclusions on tests I did with extremely noisy source files. After going back and doing some tests on DVD source files I have discovered that I can in fact get excellent quality and good file sizes using 95%+ quality. At 75% they don't look as good as my 2-pass files, but they are much smaller. At 95% they are still smaller than my 2-pass files. So now I'm going to test some at 100% and see if my original estimations of the proper file sizes for these rips was right. So I guess I still have some things to learn. Dammit I hate being wrong!

[manono]

Actually, I should apologize to you. I was thinking about it later on, and decided I was way too harsh. So, sorry. Your's was a perfectly legitimate question, and one that's been asked many times before. There's nothing at all wrong with 2-pass, since most people use it with good results. Most people want a fixed final file size. 1-pass quality mode is most useful for people storing movies on the Hard Drive to be used as a media server or HTPC. Because of the way AutoGK works, though, it's going to make some major changes if the compress test finds that the resulting quality will be too low. The changes it makes can greatly affect the compressibility, and a 1-pass can't usually be compared to a 2-pass where those changes have been made.

[electrongunner]

No worries, I was a bit of a jerk on Snapstream board myself. It sucks when you've been doing something for 5 years and think you know what you know, only to discover you didn't know what you thought you knew. I hate it when that happens.

Anyway, I think I'm starting to see the differences between 1-pass and 2-pass in terms of the output quality, though I still can't say I understand how it can perform as well as seems to. I've done the same set of 4 different STNG episodes about 5 times now. I'm keeping the 5 channel AC3 soundtracks (approx 150MB) and have ripped them the following ways:

(note: I based my standard 2-pass target file size on what I estimated I'd need to get a perfect 720 pixel wide rip with 5-channel AC3 audio out of a 45 minute STNG episode at 4:3 aspect ratio. Obviously this would lower for letterbox)

2-pass DivX Pro 5.2 compressed to 820MB (my standard)
2-pass XviD (latest bundled with AutoGK) compressed to 820MB
1-pass XviD @ 75% resulting in an average file size of about 350-450MB
1-pass XviD @ 95% resulting in a range from 473MB-610MB
1-pass XviD @ 100% resuling in a range from 1,062MB-1,539MB (my reference for 1-pass)

Using the DivX Pro 5.2 2-pass rips as my standard, it looks like the 95% single pass rips are about the closest in terms of quality. Since the recent DivX upgrades I'm no longer sure whether I like DivX or XviD better, but since I'm getting smaller file sizes, I'm thinking of switching to 1-pass rips for my DVD rips and continuing to do 2-pass compression on my more noisy HTPC mpeg2 files. 1-pass is just sooo much faster. Do you have any thoughts on what percentage is really needed to get a nearly perfect rip? Is there any reason to go higher 95%? I also did a batch at 80% and I could still see some compression artifacts in solid colored backgrounds so I just jumped up to 95% to see what the file sizes would be. I'm really looking for archival quality here. I don't mind going a bit overboard on the bitrate since I'm burning everything to DVD+R. Anyway, I welcome your advice.

[manono]

Hi-

I hate it when that happens.

Me too. Maybe everyone does. It reminds me of something I'd rather forget, along similar lines, except your response was better than mine. I'm visiting another board and see a guy beating up mercilessly on another guy. I feel sorry for him, and, anyway, I agree with him on some finer point of DVD encoding. So I jump into the fray, and the guy switches his attention to me. We battle back and forth for a couple of days, and then I start to wonder about some of the things he's saying. I do some research and begin to think that maybe he's right after all. So I come back to this board and PM a friend who knows more about the subject than I. He says I'm wrong. It's like being hit on the head with a hammer. I'm so embarrassed that I don't go back to the other board and apologize. A little knowledge is a dangerous thing, eh? Live and learn.

Do you have any thoughts on what percentage is really needed to get a nearly perfect rip?

Well, nearly perfect is quant 2 (except for any B-Frames being used). To get that you use 100%. For me viewing on a good HDTV, quant 3 is good enough. I can see where a quant 3 rip isn't quite DVD quality, but then neither is a quant 2 rip. Any time you reencode there's a quality loss. To get a quant 3 rip, you use a 65-70% figure (maybe as low as 60% these days, not really sure). Anything between 65 and 100% uses a mix of quant 2 and 3 (again, excepting the B-Frames), with higher percentages of quant 2 frames the closer you get to 100%. A lot of this is purely subjective, but my opinion is that anything higher than about 80% doesn't really take full advantage of the MPEG-4 compression ability. And that's why I think that the 75% default is a good one.

I also did a batch at 80% and I could still see some compression artifacts in solid colored backgrounds

Yes, that's one of the ways that MPEG-4 doesn't really equal MPEG-2 yet -- colored moving blocks in solid colored areas. Throwing bits at it helps some, but doesn't really solve the problem. MPEG-4 also has problems with such things as fog and smoke, black areas (the so-called "black blocks"), and compressing film grain and noise. I think that XviD is better at these things than is DivX, but since I haven't tried the new DivX, I shouldn't really say that, I guess.

Since you're the type that likes to experiment, you might pay a visit to the XviD and AviSynth forums and learn about such things as Custom Quant Matrices, new sharpening functions, deblockers and the like. It'll mean dropping AutoGK, but it may help your encoding. For example, have a look at these pics:

http://forum.doom9.org/showthread.php?s=&threadid=87514&perpage=20&pagenumber=2

[electrongunner]

Thanks for the info. I actually used to rip all my stuff the long way, but since the qualilty of progs like AutoGK/Dr.Divx and Vidomi has gotten so good, I've decided that it's no longer worth all the trouble to spend 24hours ripping a movie piece by piece. I've probably followed every guide on this site at one time or another. I think I'm gonna stick with XviD at 80-90% for a while and see how it works out. I'm just getting too old for the marathon ripping sessions. In fact, I've really started to love my DVDShrink (www.dvdshrink.org) lately for just copying movies off DVD. Most of the time I don't need to do any transcoding if I'm just copying a movie with no extras, but that's another story entirely. I can't wait for dual layer DVDs to get cheap enough to make this discussion irrelevant!

[www.ADream.de]

Encoding noisy sources:
========================

I'ven encountered the same with nearly all sources from TV Cameras.

Encoding these with a fixed quantizer (aka quality rate) of say 3 or 4 really gives HUGHE files while setting a constant bitrate of say 1200 causes high quantizers and macroblocks.

Encoding these noisy source in 2 pass mode trying to get my usual bitrate of about +/- 1200kbit, gives you a file of the desired size but the codec again uses high quantizer (up to 12 and more) to encode all the noise whil keeping the desired bitrate.

So this is also reducing the visual quality of the file and is not a solution for encoding noisy sources.

Meanwhile I achieve good results when using noisefilters in the encoding process, the one buld into DivX pro does a pretty good job in "normal position" or you use the even superior Avisynth filtes provided by GordianKnot.

[nerys]

Ok. I hate 2 pass it just takes too long! Ouch. how do I FORCE it to use single pass but also specify the file size (OR the bitrate)

I am not to worried about quality but I am doing varying length films and I need them to be smaller than 1900MB and be SINGLE pass.

how do I accomplish this? IE any other settings I can alter to increase speed even at the cost of quality.

Thanks!

[nixo]

Ok. I hate 2 pass it just takes too long! Ouch. how do I FORCE it to use single pass but also specify the file size (OR the bitrate)

Not possible in AGK.


I am not to worried about quality but I am doing varying length films and I need them to be smaller than 1900MB and be SINGLE pass.

how do I accomplish this? IE any other settings I can alter to increase speed even at the cost of quality.

Yes. Force a low resolution.

[CWR03]

how do I FORCE it to use single pass but also specify the file size (OR the bitrate)
You can achieve better quality and exact file sizes with many other apps, even with Gordian Knot (though it requires VfW build of Xvid or x264 to work right).