Keyframes in AutoGK / Xvid / DivX511

[r3mix]

I am making a large collection of maximum quality rips using AutoGK and I am having a reocurring problem with keyframes that I cannot understand and was hoping I could get some info on.

Case in point... I've used half a DVD @ 2230mb for the 107min Equilibrium because its got a lot of dark/smokey colour blending and has lots of action/flashing that I want flawless. Result was flawless quality but it had the problem where it has missed a scene change keyframe and you get a few seconds of awful blocky picture before it resets.

My questions are these...
1. Do the "scene change threshold" and "maximum keyframe" settings set in each codec via virtualdub get used by AutoGK encodes or does it do its own thing and reset them to whatever it decides ?

2. Is playing around with these DivX settings the answer and if so what am I looking for if I am doing 1/3 DVD for most rips with a target BPP of .25+ ?

3. In very high quality Xvid encodes AutoGK already switches b-frames off so why is it I get a 150,000 frame movie but still only 2743 keyframes according to virtualdub ? No problem with the results but am I missing something or shouldnt it be 1:1 if theres no b-frames ?

Thanks for any advice you can give. I got no help on Videohelp.com and I really want to get this sorted so I can get on with ripping these and not have to re-do them.

[len0x]

I have not heard of codec producing garbage in between key frames, so its most probably a playback problem (different decoder/player can help)

P.S. There are I, P and B frames. I frames are just seek positions in video structure and do not have any video info in them (maximum distance of key frames set by AutoGK is 10 sec (or 250/240 frames depending on fps)). If you switch B frames off you should end up with only I and P frames.

[stephanV]

Originally posted by len0x
I frames are just seek positions in video structure and do not have any video info in them

You might reconsider this... without the video info in intra-frames we wouldnt be seeing much video at all. :)

[len0x]

Originally posted by stephanV
You might reconsider this... without the video info in intra-frames we wouldnt be seeing much video at all. :)

May be not for all video formats?
I remember seeing in vdubmod log that key frames have 0 length of video inside them (when encoding in xvid/divx). Or am I wrong here as well?

[len0x]

actually no, key frame is really much bigger than other ones (I'm really tired you see - and my memory failing me :) )

[jggimi]

GOPs: I, P, and B frames explained (http://forum.doom9.org/showthread.php?s=&threadid=19436)

[len0x]

Originally posted by jggimi
GOPs: I, P, and B frames explained (http://forum.doom9.org/showthread.php?s=&threadid=19436)

I find the whole thread really confusing when it comes to mpeg4. Its common to have key frames (are they really I-frames?) distance more than 200 frames, which makes it very large GOPs while in that thread it is said that GOP distance is ideally quite small...

[stephanV]

i agree, that thread is a mess.

yes, key frames are really intra frames. intra frames can be decoded without having to decode a frame prior or after that, and thus use no prediction. because of that intra frames are usually the largest frames.

the GOPs they were discussing there were MPEG1/2, which has IIRC a fixed size. MPEG4 GOP-size is adaptive.

[jggimi]

Yes, an I-frame is a keyframe.

You might like the description of frame types that begins on Page 26 of the DivX 5.2 User Guide (www.divx.com/divx/divxpro/guides).

[len0x]

All this make me wonder about another thing: minimal I-frame distance. Sometimes you have 15-20 I-frames in a row on a fast motion scene. At least XviD has an option of reducing quality of consequent I-frames, but still wouldn't it be better to use delta-frames instead (I assume that reducing the quality means increasing quantizer, its probably better idea to have the same quantizer for delta frame instead).

[stephanV]

its impossible to tell, there has to be something to predict for p and b frames to work (and in high mostion scenes prediction works less to not at all). otherwise youll just get crappy video. p-frames can also hold intra-blocks (non-predicted blocks) but i believe storing too much intra-blocks in a p-frame is not not much of an advantage size-wise.

[len0x]

I thought delta frames store difference to previous frame, so in the worst case (everything is different) you have it as large as I-frame would be. But since statistically its impossible to be completely different then in most cases we would gain some bits out of it (of course not as much as delta frames on more static scenes), but without 20% loss of quality like XviD does by default. Am I missing something?

[r3mix]

Originally posted by len0x
Am I missing something?

your very own thread.
11 posts and my query hasnt even been considered let alone addressed.

reading the FAQs was a requirement of me posting. the information you are getting wrong and questioning is in them.

Originally posted by stephanV
i agree, that thread is a mess.

this one also.

[len0x]

There is no reason to be upset - all you had to do is politely ask for clarification of answers if you didn't get them (everything was answered to some extent), but instead you have to live with your problems on your own probably.

Have a nice day.

[manono]

Yeah, I thought you were kind of rude also, R3mix. len0x is right. Although they kind of beat around the bush and didn't answer your questions directly, most of that discussion was sort of indirectly centered on your questions. I'll have a go at it.

1. I think the XviD scene change detection is hard coded into the codec, so it's not adjustable. I don't know what AutoGK uses for DivX, but I assume it's the default. I haven't had problems with XviD detecting scene changes. DivX, with the default settings, used to miss them sometimes, but I don't know if it's improved since I last used it. As for maximum distance between keyframes, I assume that AutoGK uses the standard recommended settings. Those would be 10 times the framerate. 240 for 23.976fps, 250 for 25fps, and 300 for 29.97fps. I don't know what it uses for minimums, but if you have quick strobing lights, or very fast edits, then it's very possible that they don't get set every time.

2. There's no point in playing around with the settings, as AutoGK will override them anyway. If you wish to have your own settings, then you might be better off using either GKnot, or encoding directly in VDubMod.

3. I think they answered that one. You say you have 2743 keyframes (I-Frames) out of 150,000 or so frames altogether (the rest being P-Frames, in the absence of B-Frames). This is better than 1.8% keyframes, which is a pretty high percentage. Most of my movies have less than 1% keyframes.

If you got blockiness in one section, I would first check to make sure it's not in the source vobs. Many times I've found blockiness myself, and found out that not enough bitrate was assigned to that part of the source DVD. Next I would open the movie AVI in VDubMod and check if you still see the blockiness. I suspect that the blockiness is in the AVI as you say, and if a keyframe wasn't set when it was supposed to, then that might account for it. But scenes such as you describe are very difficult to encode. I'm thinking of night club scenes in a couple of movies where I've noticed the same thing, when you have strobing lights, and fast edits. You might open the AVI in the graph portion of DRFAnalyzer and check what the quants are for that problem portion. Even quant 3 might produce blockiness in such complex scenes as you're describing. When you have some frames of near black, and then a bright flash, it seems that the codec sometimes doesn't adjust quickly enough to the increased complexity, resulting in blocks.

[r3mix]

Originally posted by len0x
There is no reason to be upset - all you had to do is politely ask for clarification of answers if you didn't get them (everything was answered to some extent),

i am not upset in the least.
your first post stated that you have no knowledge of any such problem with keyframes and didnt actually know what they were for. since both of these topics are addressed in the FAQs and the next 11 posts dont address my queries and are devoted to helping you understand frame types telling you to read the FAQs was the most appropriate and useful advice for you.

if you or anyone else here finds this to be rude advice id expect you to take down all the "read the FAQ before posting" advice.

but instead you have to live with your problems on your own probably. Have a nice day.

so no change then.

[r3mix]

thanks for the info manono.
i did want to clarify some things on this topic but given the gay little warning ive been given in a PM for referring someone to a FAQ which they obviously require i really cant be stuffed.

must say i do find all this quite odd given the identity.

[len0x]

just as I suspected some people just can't be civil enough to behave properly. r3mix has now a month to think about why he shouldn't have attacked author of the program he's using (both in public and PM).

[stephanV]

Originally posted by len0x
Am I missing something?

Yes, you're overestimating the usefulness and quality of prediction.

Was it really necessary to suspend that guy? he had a point somehow...

[len0x]

Originally posted by stephanV
Yes, you're overestimating the usefulness and quality of prediction.


meaning? :)

Originally posted by stephanV
Was it really necessary to suspend that guy? he had a point somehow...

There is a difference between just having a point and expressing it properly. The guy's PM was just too much to tolerate for me especially as developer of AutoGK (its about the right attitude and has nothing to do with me being upset when he sent me reading FAQs, depsite what he thinks it was).