XviD Stats Viewer: Analyser for 1st pass log

[NiTroGen]

After seeing & using PerfectXvid by TheUnforgiven & StatsReader by Koepi, I've decided to make a graphic viewer of XviD stats file that shows a frame size distribution graph and suggests the settings for alt. curve aggression. Please tell me your opinion about my program.

EDIT: Current version is 1.65. Check page 4 of this post...

[BiaTch 5.0]

I hope this work well, I'm too dumb to understand PerfectXvid by myself &no one wants to make a explanation thread for newbies.

[NiTroGen]

Well, I'll be honest. I haven't exactly understood PerfectXviD. I think that the most important is the frame size distribution related to the average frame size, so I've made a graph of it. This program can analyse a stats file, while it's being created, without creating problems to encoding procedure. That's why I added the "Refresh" button.

Oh, I forgot to mention that my program needs the runtime libraries of Visual Basic 6.

[Maurizio]

Fine little program !
can we know how the suggested values are calculated ?
regards
Maurizio

[NiTroGen]

That's easy. High and low distances from average are the maximum and the minimum framesizes related to the average frame size. You can see this at the picture below:

(made by FiW, posted on another thread)

The curve aggression is suggested according to the related difference between the number of frames with bigger size than the average and the ones with smaller size. If there is a small difference then "Medium" is suggested. If there are more frames with bigger size than the average, then "Low" is suggested and vice versa.

[Koepi]

Ouch, that's a strange criteria for low, medium and high aggression.

The aggression just determines how fast overflow gets redistributed, so that attempt is flawed.

You most likely want a linear scaled curve as it gives the most constant quality possible with xvid. But see the statsreader thread for further details.

Regards,
Koepi

[NiTroGen]

OK, I'll try to understand better the curve aggression by reading the StatsReader thread. Thanks...

[Koepi]

It's unlikely to find out about curve aggression there You'll find the settings how to linear scale the curve

Regards,
Koepi

[NiTroGen]

Now I have removed the suggestion about the curve aggression type, but I have added a new graph of framesize related to frame number than maybe can help you predict it. Also, now the program can ignore the ending frames.

[NiTroGen]

I've changed the GUI to take less screen space and added an option to create graphs after file opening or refreshing. Here is the new executable.

Note: TABCTL32.ocx and COMDLG32.OCX are not included with this package.

[Maurizio]

After installing, when I start the program, it goes searching
for a file called data1.msi, that I don't have.
If I ignore the message, the program starts, but when i try
to open the stats file it gives runtime error 429
(activex component cannot create the object)
I'm trying on win2k professional, 5.0.2195, no service pack.
Any suggestion ?

[NiTroGen]

You probably don't have the required files: TABCTL32.ocx and COMDLG32.OCX. I've attached them here.

From now on, I won't include TABCTL32.ocx (COMDLG32.OCX is not needed anymore) in the setup package, because of the attachent size limit, but you can always find it here.

[NiTroGen]

Somebody stop me!!! Another update in a few hours.

Some bugs are fixed and a new feature is added: 2nd Pass Simulation. Well, don't ask about the method I used to simulate 2nd pass, because it is NOT right at all, but the result is very similar to the one obtained by Milan's ffvfw simulation. So, you can have a "misty" look of the final video quality. I'm still trying to find a more accurate method for the simulation...

[NiTroGen]

Now the 2nd pass simulation is more accurate than before. That doesn't mean it's perfect, but the obtained results are closer to the real ones. That's because of some options I added in the simulation tab, such as quantizer limits and I-frame % boost. You can see now the final size of the video using the simulation results, and a quality percentage of the final video. I also added some more info about the stats file. COMMDLG32.OCX is not needed any more, because of some API calls, but you still need TABCTL32.OCX.

I'm waiting for your comments, to make this program better...

[NiTroGen]

I've just released version 1.10 beta (there will always be the beta suffix, until the methods used in my program are approved by the XviD community). There are MANY bug fixes and a some new features added. Now you can ignore the last 1% to 5% of maximum framesizes in the framesize distribution, so the high distance value won't be set too high. Also you can copy the high and low distance settings directly to the registry, so XviD will have them the next time you'll open the configuration dialog.

And last but not least: A motion distribution graph is added. Motion is calculated as the summary of k-blocks and m-blocks in a delta frame. In plain english, it's the part of the frame that has changed, compairing it to the previous keyframe. This motion distribution is used to suggest the curve aggression type.

@Koepi:
I think this method is good to suggest a curve aggression type, but I'm not sure if the motion calculation algorythm is correct. Using your Nandub Stats Reader, I saw that you define motion as the number of k-blocks, but I think that m-blocks should be considered as motion, too. Anyway, I'm waiting for your suggestions about this "motion" thing and the curve aggression type.

[Koepi]

Again, flawed attempt, sorry.

take a look at your debugview output during 1st pass encoding and see why you'll always get 100% motion.

The nandub statsreader was never really finished, and if you'd read the threads back then you'd know that I was too lazy to implement the nandub way of motion calculation, which is far more complex than just showing the amount of any block types.

[NiTroGen]

Originally posted by Koepi
Take a look at your debugview output during 1st pass encoding and see why you'll always get 100% motion.
.

No, I do not always get 100% motion. Motion varies from 0 to 100%. I don't know why you say that.

About the curve aggression; I have read in this forum that a movie with many scenes of high motion should get low aggression and vice versa. So to suggest the appropriate curve aggression type we must do a motion distribution. But the real problem is "how to calculate motion". I do not agree with the nandub way of calculation (though I'm trying to understand and implement it in future versions of stats viewer), because it does not take into account the m-blocks, which, I believe, contribute as much as the keyblocks in the motion value.

[Koepi]

See, and that's why your attempt is flawed:
This is about mathematics, and not believe.

But: _believe_ me, you just want to respect kblocks for motion calculation.

example (taken from fixed quant credits):
[1016] 1st-pass: size:2247 total-kbytes:1732591 inter quant:20 H.263 kblocks:328 mblocks:346
[1016] 1st-pass: size:2651 total-kbytes:1732593 inter quant:20 H.263 kblocks:332 mblocks:348
[1016] 1st-pass: size:2742 total-kbytes:1732596 inter quant:20 H.263 kblocks:276 mblocks:402

that'd even in the credits mean 100% motion.

During the movie it's even worse, I always have about 680mblocks, and if this value drops, it's the kblocks added to the mblocks which fill the 680.... Skipped blocks are very seldom.

So, what does this mean?

I spent too much time explaining you why your program/attempt is flawed, you deny that, I explain again,...

This is my last post here as I now feel somewhat abused.

Write whatever you like, but I won't recommand that to anyone as you make impropper assumptions, don't listen to corrections of those,...

Wasted time.

[NiTroGen]

Originally posted by Koepi
I spent too much time explaining you why your program/attempt is flawed, you deny that, I explain again,...

This is my last post here as I now feel somewhat abused.

Write whatever you like, but I won't recommand that to anyone as you make impropper assumptions, don't listen to corrections of those,...

Wasted time.

I'm sorry, Koepi, I didn't want you to feel abused nor make you angry. I do not deny that my attempt is flawed. But I am really trying to learn from my mistakes and correct them. I just want to make a software that will help me, you and anybody else, to find the optimal curve aggression settings for XviD encoding. I have many questions and I need as much help as I can get. Please, I know that I'm wasting your time, but just pointing my mistakes is not enough, I need to know the right way to correct them.

Has anyone the good will to answer some questions that I have?
- Does the curve aggression type depend on the motion distribution? If not, then to what does it depend on?
- Is there a way to calculate the motion value of a frame? Is the nandub method accurate?

[Koepi]

Hi again,

hopefully you won't ignore my proposals here again

XviD has nothing which would be close to nandub's motion detection engine.

Curve aggression hasn't changed and thus determines

a) how fast overflow gets compensated (i wrote that earlier already!) and
b) due to a cosine/sine bias of the curve, whether you prefer huge frame higher quantized than small frames or vice versa.

From my experience a) is much more "visible" effect than b).

So it's best to leave that at medium aggression. If you get problems with your quantizer distribution, you can try fidling around with that, but that's not easy to see before having done a second pass.

Since you get the most constant quality using a linear scaled curve I'd suggest not to fiddle around with altCC settings too much, as they can do real harm to your encode when setting impropper values.

If you find a nice high/low value that's ok. You could set a min. req. quality or the strength so that you have a "smooth" high/lowpass behaviour.

Everything else sounds like an adventure to me

Regards,
Koepi