Need a "how to" for encoding hdtv 720p to xvid with custom matrix

[Beave]

I was reading different posts for a few hours, but I'm still not sure how to encode my Episodes to XVID. Here is what I desire:

Source:
1080i or 720p (sometimes noisy, sometimes clean)

Target:
XVID with final size of 1/3 DVD=1493mb incl. the ac3.
The Episodes are usually 43 min long so the bitrate is around 4100. My final resolution should be 1280x720.

The avs usually look like this:

1080i:
mpeg2source("1080i.d2v",cpu=4,idct=6)
Telecide(order=1,guide=1,post=2,vthresh=30)
Decimate()
crop(0,0,0,-8)
lanczosresize(1280,720)
RemoveGrain(mode=2)
ColorMatrix("Rec.709->Rec.601",mmx=true,hints=false)

720p:
mpeg2source("720p.d2v",cpu=4,idct=6)
SelectEven()
Decimate()
RemoveGrain(mode=2)
ColorMatrix("Rec.709->Rec.601",mmx=true,hints=false)

What I got so far:
I should use sixofnine hvs with first pass at quant=3.
But here is my biggest problem. I'm not sure I'm doing it right. When I tried AutoGK it sets the "Quantizer restrictions" under "Advance Options" to min=2 and max=3 and checks "Trellis quantization". Should I rather load the Xvid defaults (min=1 and max=31) and start from there?

So lets say I load the default settings in XVID. All I need to do is:
first pass:
click on "Zone Options..." and set "Quantizer: 3.00"
click on "more..." at Profile @ Level (unrestricted) and choose MPEG-Custom and load sixofnine hvs matrix

second pass:
choose Twopass - 2nd pass
fill the target bitrate
set "Zone Options..." back to "weight: 1.00"

Is that all? Nothing else? Can I leave all the default setting or did I forget to change something?

What about my compressebility tests in enc? Do I set it to "single pass" and Target quantizer: 3.00 or do I use the Zone Options again? Setting it to "Twopass - 1st pass" with Zone = quant:3 delivered an error in enc. When I use my typical enc setting of single pass quant=2 I get a compressibility of around 50% - 75% with sixofnine hvs. That should be pretty good considering quant=2, right?

Any help is greatly appreciated, even if it involves another matrix or how to speed up the first pass without loosing quality. BTW. I'm using the XVID version integrated in autogk 2.08b.

[Didée]

Yes, "that's all. Nothing else". ;)

Regarding XviD defaults - you might want to give a go on additionally activating Qpel, VHQ2~4, and VHQ for Bframes. B-VHQ in any case, I'd say.

For size prediction through enc, you've to set up single pass @ fixed quantizer (in zone).

And a comp.check @ 50%-75% should be very comfortable, for SixOfNine @ q2. Using q3, the comp.check should be comparable to a "usual" comp.check.

For episodes that need more bitrate than expected, perhaps try LRemoveDust.

[Beave]

Thank You,
that is really helpful. I tried an Episode in 720p in enc and got:

55% with 6of9 @ q2
98% with 6of9 @ q3
72% with eqm_v3hr

I normally use comptest result of 65%-80%. The difference is pretty big here between 55% and 98%. Would this mean I could savely remove RemoveGrain() and/or CPU=4 in my avs? (Although I don't mind both filters, I like less blocks and RemoveGrain is hard to spot anyway)

Which VHQ would You suggest of 2~4? Probably as much as my patience permits...;)
I think I read somewhere, that Qpel and GMC are not very usefull at high resolutions, but you suggested Qpel. I will give it a try.

The Quantizer restrictions min=1 and max=31 are ok then, right? I'm only asking directly because AutoGK used min=2 and max=3. Since You did not mention it, I assume XVID with 6of9-hvs will choose the right quantizers without restricting it.

[Didée]

CPU=4, hummh.

If compressability is that good, you could as well do 1-pass CBR encodings with target bitrate, and restrict all quantizers to 2,3. (And restrict minimum quants always to "2", never to "1". An unlucky default that is.)

For higher quantized 2-pass encodings, with targeted average quantizers (6of9) of 3.5 to 5.0, I tend to cap quants to min = 2, max = [avg.quant * 1.66], and use some slight curve compression (for these cappings, e.g. CC = low=15 & high=30)

Qpel of course is not a must at that resolution and low quantizers. Just try with and without, and decide by the look of both.

Oh, and I think for your 1080i sources, you should also specify "ipp=true" in Mpeg2Source, for PP to treat the interlaced content correctly.

[Beave]

CPU=4, hummh.
You wouldn't do that?

For higher quantized 2-pass encodings, with targeted average quantizers (6of9) of 3.5 to 5.0
How do I find out what my targeted average quantizers are? Can I see that somewhere in enc?

Oh, and I think for your 1080i sources, you should also specify "ipp=true" in Mpeg2Source, for PP to treat the interlaced content correctly.
I thought ipp is set to "auto" when not specifying it? Shouldn't it autodetect what the framesource is?

Thanks for the tips with the quantizers.

[Didée]

You wouldn't do that?
Depends on how blocky the source really is. Mostly I try to avoid it.

How do I find out what my targeted average quantizers are? Can I see that somewhere in enc?
Do a prediction of 2~3% @ q3. Prediction lower than target, make a prediction at q4. Smaller than target, make one at q5. Bigger than target -> simple math gives a target avg.quant of four-point-something, like 4.3 , or whatever. With practice, you're usually done with a most of two different predictions, and often enough you'll do only one, and estimate the rest freely.

I thought ipp is set to "auto" when not specifying it? Shouldn't it autodetect what the framesource is?
Oh, there is auto? Well, being a paranoid man, I did always set it manually. Imagine a stream is flagged interlaced, but the content actually is progressive ;)

[Beave]

I made some more test with the quantizers and now I got another question. If I get a target q = 2.9, should I still use q=3 for the first pass or is it better to choose 2.5 or maybe 2.0 as the 1st pass?

What are your recommended settings for lremovedust for hdtv sources?

[Didée]

If average quantizer is so close to 3, use q3 for the 1st pass. However, I have never made a 2-pass encoding aiming at such low quantization. After all, q3(6of9) ~~ q2(standard) ...
For that reason, I repeat: if you get an average quant smaller than 3, then you can just as well do 1-pass "CBR" encoding, with quants capped to [2,3].

For LRemoveDust, try (4,1), (4,2) and (4,3). I've no clue how much noise there is in your sources.

[Beave]

Thanks so far. It really looks great. The PQ is extremely close to the original.

I have one major problem though:
The frames in the avs are not corresponding to the frames in the final avi!

So if I look at a frame in the avs with the number 41528 it is not the same in the resulting avi. It's frame number 41611 there if try to find the matching frame. The difference of frames changes all over the place, so there is no special point where it all starts to drift apart. Some frames are dublicated, some are missing. How could that be possible? Shouldn't every frame be at the exact same number in the final encode?

Here is my script:

LoadPlugin("C:\PROGRA~1\AutoGK\DGMPGDec\DGDecode.dll")
LoadPlugin("C:\PROGRA~1\AutoGK\filters\decomb.dll")
LoadPlugin("C:\PROGRA~1\AutoGK\filters\ColorMatrix.dll")
LoadPlugin("C:\PROGRA~1\AutoGK\filters\RemoveGrainSSE2.dll")

movie = mpeg2source("payback.d2v")
Telecide(movie,order=1,guide=1,post=2).Decimate(mode=0)
crop(8,144,-8,-144)
RemoveGrain(mode=2)
ColorMatrix("Rec.709->Rec.601",mmx=true,hints=false)


After the encode I could see a few warnings about possible life locks in VDMod. Is the encoding so sensible maybe? I used to encode quite a lot (~400 movies) a few years back, but never had this problem before.

Any Ideas?

[Teegedeck]

The VDub warnings are nothing serious, you can ignore them.

About not finding the precise frames in the XviD file that you have in the AVS: VDub simply has problems with b-frames unless they are inside a 'packed bitstream'. Packed bitstream is not exactly a recommendable feature though, so I would not advise to use it unless you absolutely must edit your XviD file with VDub.

If you want to make a frame-by-frame comparison of the avs-source and the XviD file, just load the XviD as 'directshowsource("....avi")' in another avs.

[Beave]

Thanks, but what do I do about muxing in the audio? It's out of sync at different places. I tried avimux_gui, but this did not help either.

I can see that in some parts of the movie its not playing smooth in Zoom Player (ffdshow) as well.

Using directshowsource to compare actually helped, but did not solve the problem completely. Both files are much closer now, but still a few frames apart. In some places 2 in some places 5 and so on...

Maybe I should try packed bitstreamed b-frames? The last movie took me 40 hours to encode. 20 hours each pass. Takes a lot of time trying. Short samples usually don't show the problem unfortunatly.

[Didée]

Using directshowsource to compare actually helped, but did not solve the problem completely. Both files are much closer now, but still a few frames apart. In some places 2 in some places 5 and so on...
If your script contains decimate(), the only way to safely compare is linear playback from start to end. As soon as you "seek" in the avs preview, decimate will refresh the window for it's cycle based search. This window will have another alignment in respect to the actual frame numbers - kind of "random", depending on how you jumped around in the timeline - and hence might give different results as it did before in the (linear) encoding.

Did we note that first making a lossless compression from the avs script, and then doing an XviD-2-pass on *that* file, is mostly a good idea? Though that's 3 passes instread of 2, it will shorten your overall encoding time ;)

[yaz]

@beave
a silly question :
is your pc 'strong enough' for playing back such resolutions ? it seems (at the 1st glance) as u were driving on the edge. it may cause temporary synch issues and jerkyness at hi-mo scenes. try to restrict yourself ;) e.g., AS@L5 would be fine for 720p.

the bests
y

[Beave]

@yaz
thanks for the tip, but it's not a slow cpu. I have a athlon64 2800+ @1980mhz. Hope It's not the 10% overclocking. If I only encode a sample it plays smoothly

@Didée
Is it possible that decimate() chooses the wrong frames to decimate once it's reaching frame number 134862, but does the right job when I start there and only encode a small sample?

I uploaded a small sample (http://www.gustmann.com/jerky.zip) from that jerky file and the same part encoded directly from the avs.

[yaz]

...it's not a slow cpu. I have a athlon64 2800+ @1980mhz.i didn't mean only the cpu. ram, bus-speed, graphics-card may also restrict.
@Didée
Is it possible that decimate() chooses the wrong frames to decimate once it's reaching frame number 134862, but does the right job when I start there and only encode a small sample?everything's possible ;) especially when working w/such big frames. afaik, decimate keeps and handles more frames at once, so it consumes the memory very heavily. and when windows starts flip-flopping, u can never tell what comes.
use setmemmax() and grab as much memory as possible. wise-men suggest 2/3 of the total, at least.
and force frame-alignement (say, w/crop) it may also help.

the bests
y