View Full Version : new tool PerfectXviD helps to get best setting for alt-curve


TheUnforgiven
11th August 2002, 19:34
i've coded a tool that analyses the xvid first pass debubview log file and gives frames distribution around the average frame size.
here is an example of the output:

EDIT: these results r wrong see the correct results in my other post down.

Number of frames = 5797
Average frame size = 13,135
Max frame size is +376% over the average
Min frame size is -565% under the average

+400%
[ 4 frame 0.07%
+300%
[ 10 frame 0.17%
+200%
[ 261 frame 4.50%
+100%
[ 2296 frame 39.61%
--- Average frame size ---
[ 2153 frame 37.14%
-100%
[ 768 frame 13.25%
-200%
[ 163 frame 2.81%
-300%
[ 58 frame 1.00%
-400%
[ 13 frame 0.22%
-500%
[ 1 frame 0.02%
-600%

i think these results have the potential to help us decide the best settings for the alt-curve for any movie but i need ur help guys to test and find the best way to decide alt-curve settings based on info similar to this and i want to know ur opinion also if this is a good idea to decide the settings based on such analysis.
about the program it is called PerfectXviD and i'll post it soon.

unplugged
11th August 2002, 20:14
This is very good reference point (to mention one thing... Gordian Knot give frames size stats considering also keyframes? That was a problem!), I think some of that information should be implemented in XviD config GUI :)

But aren't tools like DBG or ALT-DBG doing the ~same thing yet?
Why not consider also the stats file as source, Gordian Gnot is cool but lacks something...

Originally posted by TheUnforgiven
Min frame size is -565-% under the average

:scared: Aaaargh! This is very misleading, please correct this calc. policy or many non experienced dudes could get drilled by their params choiches... :D

P.S.: important, please report (and specity this) all frame size statistics firstly withot keyframes (treat them separately).

Didée
11th August 2002, 20:24
[edit: unplugged seems to be much quicker in typing than me ;) ]

This could evolve into something really useful. I´ll leave the discussion to the - more experienced ;)

But, one first comment:

Something´s wrong with your "below average framesizes" !

"average framesize" = 100% --> "100% below average" = 0 kByte framesize

The fact you can enter values > 100% for "low distance" in the configuration, deals only for scaling the curve (have a look at foxer´s graph).

Besides that, things get more confusing to me, lately.
Default is Lo300/Hi300/MidAggression/Strenght50%.
Somehow, Lo100/Hi300/LoAggression/Strenght50% seems to be kind of universal weapon.
Now, lately I read several posts where Koepi (and others) talked about
Lo90/Hi500(!)/MidAggression/Strenght30%(!), without further explanation. This differs significantly from what´s common ...

I really love this great codec, but I have a hell lotta work in "real life", and therefore have no time to play around, play around, play around ... and would really like to see some more solid guidelines.

Perhaps this tool you are beginning to invent gives a kick in that direction?

TheUnforgiven
11th August 2002, 20:54
@unplugged
-i think dbg and alt-debug r meant to analyse the second pass debug info to give quantizers info. my prog analyses first pass to help deciding alt-curve settings.
-the misprint corrected now
-about keyframes i think they r included cuz i just processed everything xvid debug prin has thrown i dont know the effect of this till now.

@Didee
EDIT: i'll think again and i'll have a look on the code to see y this is happenning.

@all this tool is early beta and maybe it has errors on calculations or in some concepts but the important thing is the idea and how can we develop it to get better alt-curve results

unplugged
11th August 2002, 21:20
Originally posted by TheUnforgiven
-about keyframes i think they r included cuz i just processed everything xvid debug prin has...

Identify them, it's important for your tool

Originally posted by TheUnforgiven
...i dont know the effect of this till now.

Simply the reason:
1) keyframes have out of control *size gain*
2) You don't and can't control them from XviD alt-curve menu (of course)

so it's useful to keep out keyframes (I-Frame) from the main statistics since they are nothing to do with curve control, the main target of your tool.

Good work ;)

OUTPinged_
12th August 2002, 11:49
Didee,

The "strength" setting is pretty useless if you dont know 1/2pass ratio. Set MRQ instead.


unforgiven:

Quick and probably incorrect way of guessing decent 2pass settings:

1. take only delta sizes excluding credits.
2. calc averages in increments of 10% and average framesize.
3. look at maximum quantizer, get mrq/strength values from it.
4. look ar averages. there are frames that have "unnaturally high" framesizes. They are pretty easy to detect and should be excluded from decision where to place a high distance.
5. value for a low distance should have a more strict threshold for low dist. decision.
6. The tool should give a preview at how quantizers will be distributed. Newbies will gain a lot from tweaking values and instantly checking if they are any good or not.

(The default values are really not something you should use even if you are a newbie. If i could, i would suggest Foxer to make other values as a default for a quick encoding)

sierrafoxtrot
12th August 2002, 12:16
here's my 2 pence,

if you could have an equivalent of GKnot's nandub stats analysis function (graphing 1st pass size vs calculated 2nd pass) it would be fantastic. that way you could throw loads of different high low distances at the it, and it would graphically display the projected bitrate distribution for each frame.

i also agree with outpinged about unaturally large frames distorting the calculation. i guess you should really weight the frame size distribution, ie, if you've got 200 frames of >70,000kB, then they get less wieghting in the calculation.

am i making any sense?

;)

TheUnforgiven
12th August 2002, 14:44
the bug pointed by Didee is fixed now here is the correct output of the example in the first post:

Number of frames = 5797
Average frame size = 13,135
Max frame size is +376% over the average
Min frame size is -85% under the average

+400%
[ 4 frame 0.07%
+300%
[ 10 frame 0.17%
+200%
[ 261 frame 4.50%
+100%
[ 2296 frame 39.61%
--- Average frame size ---
[ 3177 frame 54.80%
-100%

TheUnforgiven
12th August 2002, 14:56
the tool is attached to this post i hope they'll approve it quickly
ver 0.2

EDIT: newer version is available below this one was removed

TheUnforgiven
12th August 2002, 15:10
@unplugged, outpinged, sierrafoxtrot, didee:
thanx for ur great suggestions i'll study them carefully.
but the big question now is how can we tweak the alt. curve settings according to the simple info provided by the tool. i want to see the tweaking possibility before going further.
for example can someone suggest alt. curve settings with explanation for the video clip analysed above.

i see the point about i-frames and i agree with u but the tool now is just analysing debug info for the first frame which doe not have any info about frame type maybe we can do some hack in the future but let us see some results first to see if the tool is good anyway to enhance our encodes.

sierrafoxtrot
12th August 2002, 15:24
thanks for writing this, i'll test it the moment the attachment is approved.

i'm not too sure about what alt curve values would work for the particular clip, but imho, i think that i would use hi/lo 200/100, because there are very few frames > 300% and the majority of frames are +/-100%.

actually, come to think of it, all these calculations don't take into account payback rate (bias or proportional) and strength (or does it?) and i *know* strength has a massive effect on quality. i'm just happy to have another tool to mess about with ;)

iago
12th August 2002, 18:42
I totally agree with Didée on his above post, (and in that we've "...got hell lotta work in real life..." ;).

And such a tool (analyzing the stats file for frames distribution with respect to average framesize) can be of great help to decide on the second pass alt.CC settings, which will (obviously) differ from movie to movie in most cases.

best regards
iago

TheUnforgiven
13th August 2002, 18:55
i don't know y the attachment did not appear till now !
anyway u can download it from here:
http://www.everwicked.com/forums/showthread.php?s=&postid=8116#post8116

and plz give feed back if u could optimize anything with it.

Acaila
13th August 2002, 22:42
I've just tried out your (very promising :)) tool, and I've come to some strange results.

This is the output for a movie I've encoded recently:
Number of frames = 136365
Average frame size = 30
Max frame size is +100% over the average
Min frame size is -100% under the average

+100%
[ 31833 frame 23,34%
+50%
[ 34058 frame 24,98%
--- Average frame size ---
[ 34134 frame 25,03%
-50%
[ 31784 frame 23,31%
-100%
It has an "average frame size" which seems a bit too small to me :) (unless it's in kb?). Also the range of the higher than average frame sizes is quite small. Could be because I cap the max quant where there are only a few frames that receive it, which means it's already highly optimized.
The movie reached an average of 2.62 for the 2nd pass btw.

Ps.
Your tool "forgets" the output it calculates when it loses focus...

iago
13th August 2002, 23:21
@Acaila

Also the range of the higher than average frame sizes is quite small. Could be because I cap the max quant where there are only a few frames that receive it, which means it's already highly optimized.

As I understand it, though I haven't tried yet, the tool is designed to work with the "1st pass" debugview log file. How come can you cap the max quant in the first pass? Isn't it always a constant quant of 2 in the first pass? :) (If so, the restriction in your file analysis might be due to sth else?)

Please correct me if there's something I got wrong as to how the tool works (doesn't it work with the first pass debugview log file?).

Thanks,
iago

Acaila
14th August 2002, 06:32
@iago:

Yes I know, it was an misunderstanding on my side :)

I remembered it as soon as I shut down my computer, and I didn't want to start it back up :D

avih
14th August 2002, 07:19
this tool and it's 'recommendations' (when done) could be a nice step towards automatic cc setting in the xvid vfw itself ;)

rui
14th August 2002, 09:59
OK, i want to congratulate TheUnforgiven for giving us such a great tool!

In spite i like the possibilities that alt cc. gives to the user, in the last days i have used default cc many times. Now, using your tool, i managed to lower the average quantizer of a test i made, both comparing to default cc, and using "normal" alt cc. :)

Since no one spicified on this, i used medium agression and left strength at 50%.

TheUnforgiven
14th August 2002, 15:01
@Acaila
I've come to some strange results.

i think everything is ok now, waiting for ur next test :D

Your tool "forgets" the output it calculates when it loses focus...

thats true i'll fix that in future versions.

@iago
Please correct me if there's something I got wrong as to how the tool works (doesn't it work with the first pass debugview log file?).

thats exacly what it is.

@avih
this tool and it's 'recommendations' (when done) could be a nice step towards automatic cc setting in the xvid vfw itself

i hope that we can find the good recommendation through the discussion and tests here.

@rui
Now, using your tool, i managed to lower the average quantizer of a test i made, both comparing to default cc, and using "normal" alt cc.

very happy to hear that and thanx for the congratulations.

Acaila
14th August 2002, 15:12
i think everything is ok now, waiting for ur next testWell, not exactly :). Can you explain why the average frame size is only 30?? (I believe it's in bytes, but can you confirm that?)

TheUnforgiven
14th August 2002, 15:33
@Acaila
did u use the f-pass only debug file?
if u used the one with 2-pass info also the prog will behave incorrectly, maybe the prog thought there r much more frames because u load a debug file with extra lines(info) other than the first pass info.
have a look at the example .LOG file that comes with the tool, the input must be like this one.

rui
14th August 2002, 17:13
Just one small thing: when i choose the interval to be 50%, one must creat an outside file for the report, because the amount of data displayed is higher than the space in the PerfectXvid software window, at least in my case (800x600 resolution), and i couldn’t find a way of scrooling down.

But not big problem, because i can always create the txt with all the data.

Then i have a small doubt: a report of a test i made mentions:

–75% of average frame size –> 43,64%

With this value, i should choose, for low motion, 25 or 75%?
I am asking this because, if i am correct, the report is saying that i have many frames below the average, so i should decrease the low motion to 25, so more small frames get more data.


Right? Or not? :D

And when te report mentions -100%, what value should one choose for low motion?

Sorry if i am asking something that should already be long gone (after all the alt cc thread was very big and full of explanations), but i for one admit that, after so long, i still have some doubts about alt. cc behaviour :(

iago
14th August 2002, 17:28
Hi again,

And, in addition to rui's question, here's the report of a test I made with the tool:

Number of frames = 178576
Average frame size = 7.088
Max frame size is +1.425% over the average
Min frame size is -99% under the average

+1500%
[ 1 frame 0,00%
+1400%
[ 5 frame 0,00%
+1300%
[ 7 frame 0,00%
+1200%
[ 13 frame 0,01%
+1100%
[ 12 frame 0,01%
+1000%
[ 40 frame 0,02%
+900%
[ 73 frame 0,04%
+800%
[ 181 frame 0,10%
+700%
[ 237 frame 0,13%
+600%
[ 302 frame 0,17%
+500%
[ 478 frame 0,27%
+400%
[ 1664 frame 0,93%
+300%
[ 3898 frame 2,18%
+200%
[ 12185 frame 6,82%
+100%
[ 41506 frame 23,24%
--- Average frame size ---
[ 116865 frame 65,44%
-100%


Yes, what would be your recommendations for the altCC settings according to this report? Setting low 100, high 500 or something? :)

regards,
iago

TheUnforgiven
14th August 2002, 17:37
@rui
i'll try to fix the thing with scrolling in future.
i think theoritically u should set low something a little before the 75% maybe 70%.if i am wrong plz someone correct me (i still have doubts like u).and maybe u should wait for another opinion on this but i think we all assume things noone is sure about. this is y i made this tool, to verify these things. y don't u do a test for us in different values and tell us the results.
the alt. curve topic is not over yet maybe u will find the best settings.

TheUnforgiven
14th August 2002, 17:54
@ago
really u have very big frames i doubt they r very big i-frames.i guess u have restricted i-frames quantizer very much.the problem with i-frames is discussed before and i'll try to come up with a solution to exclude them from the calculation. maybe u should ignore the very very big size frames cuz they r most probably i-frames and focus on tweaking 65% frames below the average. u can read my answer to rui but i say here that i am not the one to recommend settings because simply i am not an expert in alt. curve. and i repeat my advise in testing.

iago
14th August 2002, 19:33
@TheUnforgiven

really u have very big frames i doubt they r very big i-frames.i guess u have restricted i-frames quantizer very much.the problem with i-frames is discussed before and i'll try to come up with a solution to exclude them from the calculation
I also think that this great range is due to I-frames. Capping the quantizers is only an issue of the second pass, whereas your tool works on the first pass debugview log file and the "first pass" is done with a "constant quantizer of 2".

Imho, I-frames, which obviously lead to such great deviations, should absolutely be excluded from the calculation in your tool.

And, btw, thanks a lot for working on such a promising tool, which certainly will be more accurate and useful when the I-frames are excluded from the report file.

best regards,
iago

TheUnforgiven
15th August 2002, 15:45
i have an idea to implement i-frame guessing from the debug info but i need ur help.
in the debug info of the first pass we get the number of kblocks and mblocks, if the mblocks is more than 50% the frame will be considered i-frame.
my questions r:
1-r the blocks always 16x16?
2-what is the difference between kblocks and mblocks?
3-is this a good way to guess i-frames?


@iago
Capping the quantizers is only an issue of the second pass, whereas your tool works on the first pass
sorry i forgot that.

Acaila
15th August 2002, 18:00
Why don't you just check for intra or inter frame? Inter is P/B-frame, intra is I-frame.

rui
16th August 2002, 11:09
Well, i made a couple of small tests (i know that these are really small tests, but here i only have a P2-350, and it would be very hard to do something bigger) , but they weren’t much conclusive.
I the first pass, i got a report saying :

Number of frames = 3570
Average frame size = 20.196
Max frame size is +371% over the average
Min frame size is -99% under the average

+400%
[ 1 frame 0,03%
+350%
[ 3 frame 0,08%
+300%
[ 6 frame 0,17%
+250%
[ 16 frame 0,45%
+200%
[ 58 frame 1,62%
+150%
[ 154 frame 4,31%
+100%
[ 317 frame 8,88%
+50%
[ 827 frame 23,17%
--- Average frame size ---
[ 1590 frame 44,54%
-50%
[ 534 frame 14,96%
-100%

So i made two encodes, one using medium agression and high-250% low-75%, and another one using high-250% and low-25%

In the first one i got an average quantizer of 7,339. In the second i got an avergae quantizer of 7,346 (a little higher). But in the first test some quant 18 were used, and in the second test a max of quant 17 were used.

This leads me to another question: it’s better to have a low average quantizer or to have a little higher average quantizer but lower absolute quantizers used?

biersauf
16th August 2002, 11:38
uh, can someone explain me that? why do you get high-250% an low-25%?

rui
16th August 2002, 12:35
Because the values of 200% and above have very few frames above average. So i choose to take bits only for frames at 250% above average.
And below average, i have 44,54% frames at 50% below average, and 14,96% at 100% below average.
So i decided to choose 25 and 75 to give some margin between settings, to try to obtain some diferences. But as you can see, the diference between average quantizers was very small. Correct, it was a small test ;)

biersauf
16th August 2002, 14:31
oh my god. dont mind. can someone else explain me that :( . thank you for your explanation, but... anybody german? plssss

sierrafoxtrot
16th August 2002, 15:20
um, koepi's german, but i think he's probably got his hands full atm. i haven't had a chance to play around much with this program yet, but from what i've seen, rui's right. if you try loading your stats file in Gknot, under the nandub files tab, you should be able to click on the distribution button for framesizes. then you get a couple of nice curves, that show framesize distribution vs filesize.

here's what i think:

i) the average quantizer shouldn't change too much, you are only using alt cc to distribute the bits in a manner which gives constant quality.

ii) if you've got loads of frames smaller than av. (see iago's post in this thread) 44.54 + 14.96 = 59.5%, then you should raise the low percentage (ie, from 100% to 50%)

iii) if you've got lots of frames significantly larger than average (shot in the dark here, >10% above +300) then you should raise the hi percentage value.

iv) i think curve aggression should be adjusted based on the distribution of the framesizes. if you've got a distribution like iago's (try plotting x-axis: % deviation from av, y-axis: number of frames) then i'd go for medium aggression. but if you get a more even distribution, then i'd go for high aggression, because there are a wider range of framesizes.

sorry if that doesn't make much sense. anyone got any thoughts?

i've just gotten rumble in the bronx and donnie darko, and i know donnie darko is really compressible, and i'm guessing jackie isn't going to be :D , so i'm going to encode both over the weekend with hi/lo 500/100 and hi/lo 100/500, high and low aggression just to see what looks better and try to correlate that with PerfectXvid.

maybe someone else could try something similar. rui? ;)

TheUnforgiven
16th August 2002, 16:12
@acaila
i know u think i am stupid.. i discover it myself last night.
the new version is ready with i-frame exclusion i'll post it now

TheUnforgiven
16th August 2002, 16:19
this is a new version with everything u asked for :D

ver0.3 changes:
- i-frames are excluded now from the distribution.
- improved GUI with scroll bars.
- no need now to recalculate everything when changing the interval length.
- new interval length, 25.

it is attached to this post but if u cant wait untill it get approved u can get it here:
http://www.everwicked.com/forums/showthread.php?s=&postid=8166#post8166

TheUnforgiven
16th August 2002, 16:33
@rui
thanx for ur results
This leads me to another question: it’s better to have a low average quantizer or to have a little higher average quantizer but lower absolute quantizers used?
i think this depend on what frames u want to optimize high,low or something in between.

@sierrafoxtrot
mmm.. very interesting

rui
16th August 2002, 16:40
Originally posted by sierrafoxtrot
if you try loading your stats file in Gknot, under the nandub files tab, you should be able to click on the distribution button for framesizes. then you get a couple of nice curves, that show framesize distribution vs filesize.

Interesting. I didn't know that.
Originally posted by sierrafoxtrot

iv) i think curve aggression should be adjusted based on the distribution of the framesizes. if you've got a distribution like iago's (try plotting x-axis: % deviation from av, y-axis: number of frames) then i'd go for medium aggression. but if you get a more even distribution, then i'd go for high aggression, because there are a wider range of framesizes.


Another good point. So, in my test above, you think that i should use high agression instead of medium, since i got very few frames above the 200% mark, right?

Well, i pretend to compress my favorite "The Replacements" (very very hard movie) this weekend. I will share the expericence with you guys later.

iago
16th August 2002, 19:13
@TheUnforgiven

ver0.3 changes:
- i-frames are excluded now from the distribution.
- improved GUI with scroll bars.
- no need now to recalculate everything when changing the interval length.
- new interval length, 25. Nice work. Will give it a try ASAP. ;)

iago

unplugged
16th August 2002, 21:19
And anyone clear me/us this :(

Number of frames = 191744
190090 delta frames and 1654 key frames (excluded)
Average frame size = 8.982
Max frame size is +1.025% over the average
Min frame size is -98% under the average

+1100%
[ 3 frame 0,00%
+1000%
[ 6 frame 0,00%
+900%
[ 6 frame 0,00%
+800%
[ 17 frame 0,01%
+700%
[ 95 frame 0,05%
+600%
[ 291 frame 0,15%
+500%
[ 640 frame 0,34%
+400%
[ 1228 frame 0,65%
+300%
[ 3013 frame 1,59%
+200%
[ 7939 frame 4,18%
+100%
[ 50797 frame 26,72%
--- Average frame size ---
[ 124570 frame 65,53%
-100%

So when I set 350% in High curve (alt-CC) 2nd-pass setup

I would scale 1100% frames to 350% ??
I would scale 500% frames to 159% ??? (proportionally)
I would scale 300% frames to 95% ?!?!!! :scared:

?

biersauf
16th August 2002, 21:44
ahhh, how did you get these settings???

unplugged
16th August 2002, 22:34
Settings?

To what do you refer exactly?
I have used last version, 0.3.
350% is a common point for high CC normalizing.

Maybe PerfectXviD still includes I-Frame in statistics... :rolleyes: (bug?)

unplugged
16th August 2002, 23:01
Originally posted by unplugged
So when I set 350% in High curve (alt-CC) 2nd-pass setup

I would scale 1100% frames to 350% ??
I would scale 500% frames to 159% ??? (proportionally)
I would scale 300% frames to 95% ?!?!!!

Ummm... I will be more clear: :p

When I set 350% (ceiling) in High curve (alt-CC) 2nd-pass setup the encoder do such kind of scaling:

- Frames located to ~1100% range get normalized to ~350% (ratio 3,1428)
Thus proportionally:
- those at ~500% range get normalized to ~159%
- those at ~300% range get normalized to ~95%

iago
17th August 2002, 01:43
@unplugged
Maybe PerfectXviD still includes I-Frame in statistics... (bug?) Here are the two different results of the same first pass debugview log file: "Report1" is generated with the old 0.2 version (keyframes included), "Report2" is generated with the new 0.3 version (keyframes excluded):

Report1:

Number of frames = 214128
Average frame size = 5.548
Max frame size is +1.571% over the average
Min frame size is -98% under the average

+1600%
[ 1 frame 0,00%
+1500%
[ 0 frame 0,00%
+1400%
[ 1 frame 0,00%
+1300%
[ 0 frame 0,00%
+1200%
[ 1 frame 0,00%
+1100%
[ 6 frame 0,00%
+1000%
[ 21 frame 0,01%
+900%
[ 79 frame 0,04%
+800%
[ 140 frame 0,07%
+700%
[ 253 frame 0,12%
+600%
[ 373 frame 0,17%
+500%
[ 502 frame 0,23%
+400%
[ 1216 frame 0,57%
+300%
[ 4011 frame 1,87%
+200%
[ 14321 frame 6,69%
+100%
[ 54545 frame 25,47%
--- Average frame size ---
[ 137285 frame 64,11%
-100%


Report2:

Number of frames = 214128
202370 delta frames and 11758 key frames (excluded)
Average frame size = 5.618
Max frame size is +1.346% over the average
Min frame size is -98% under the average

+1400%
[ 1 frame 0,00%
+1300%
[ 0 frame 0,00%
+1200%
[ 0 frame 0,00%
+1100%
[ 1 frame 0,00%
+1000%
[ 7 frame 0,00%
+900%
[ 71 frame 0,04%
+800%
[ 102 frame 0,05%
+700%
[ 152 frame 0,08%
+600%
[ 222 frame 0,11%
+500%
[ 319 frame 0,16%
+400%
[ 981 frame 0,48%
+300%
[ 3614 frame 1,79%
+200%
[ 13892 frame 6,86%
+100%
[ 53747 frame 26,56%
--- Average frame size ---
[ 127894 frame 63,20%
-100%


It seems (to me) that the "exclusion of keyframes" issue is solved.

However, (interestingly or expectedly :)) the framesize distribution ranges do not differ so much in the two separate files (maybe for this movie I analyze?), whether the keyframes are included or excluded. At least the range differences between the two output files are so small and negligible that it will possibly not affect how you're gonna decide to set the Alt.CC parameters. (Probably I would decide on the "same" Alt.CC parameters, whichever of the above reports I take as a basis.) Also, this seems to me the possible result that we would get with most movies (except some really extreme cases).

Any comments and opinions?

regards,
iago

Sygma21
17th August 2002, 02:36
Number of frames = 136246
135115 delta frames and 1131 key frames (excluded)
Average frame size = 6_592
Max frame size is +1_108% over the average
Min frame size is -98% under the average

+1200%
[ 1 frame 0,00%
+1100%
[ 1 frame 0,00%
+1000%
[ 10 frame 0,01%
+900%
[ 8 frame 0,01%
+800%
[ 8 frame 0,01%
+700%
[ 83 frame 0,06%
+600%
[ 218 frame 0,16%
+500%
[ 262 frame 0,19%
+400%
[ 905 frame 0,67%
+300%
[ 1972 frame 1,46%
+200%
[ 5928 frame 4,39%
+100%
[ 41889 frame 31,00%
--- Average frame size ---
[ 82827 frame 61,30%
-100%

I read carefully the post and I suppose that best settings are Medium/Hi 250/Lo 90. Right or wrong ?

unplugged
17th August 2002, 03:09
I think that PerfectXviD is (in part) already viable for its job. Thanx goes to TheUnforgiven.

About XviD itself:

2nd-pass: quant:2 H.263 inter stats1:17164 scaled:12591 actual:19511 overflow:-3422 movie

Now, IMO the intersting point is to know *exactly* how works *numerically* the curve scaling in XviD

1st I have noticed one thing, the "scaled:12591" value from the log is partially useless (or useless) because it reflects *only* 1st-pass/2nd-pass ratio, NOT the alt-CC curve values (especially the numbers for lowest and highest limit).

So, every number you encounter in scaled field is simply stats1*2nd-pass/1st-pass (ratio).
This is not clear to know the intentions of the encoder frame by frame :mad: (sure, not the intentions of alt-CC algo! Damn!)

2nd Because I can't trust on that "scaled" value I go directly to look the "actual" value (that must reflect the encoded result).
Sadly, looking this value there are continuous fluctations that prevents me to enstabilish an average scaling ratio between "actual" and "stats1". (of course, there is a big compression jump between one quantizer and the upper :rolleyes:, in fact the encoder MUST continuously fluctuate to compensate)

:stupid:
Indepth experts needed, or better... developers.

IAmBlakeM
17th August 2002, 07:54
@Sygma21

I'd think you'd want that high range up considerably higher since you have a ton of frames above the 250 mark. I'd guess you want that level around 500%-600% or else your very big frames are going to be reduced too much.

biersauf
17th August 2002, 11:00
k, heres my sample:

Number of frames = 1253
1247 delta frames and 6 key frames (excluded)
Average frame size = 10.621
Max frame size is +221% over the average
Min frame size is -99% under the average

+300%
[ 8 frame 0,64%
+200%
[ 73 frame 5,85%
+100%
[ 475 frame 38,09%
--- Average frame size ---
[ 681 frame 54,61%
-100%

so, which settings do you prefer? can anybody explain me that? now, i would set high to 100 and low to 100. (?)

ok, same sample:

Number of frames = 1253
1247 delta frames and 6 key frames (excluded)
Average frame size = 10.621
Max frame size is +221% over the average
Min frame size is -99% under the average

+225%
[ 8 frame 0,64%
+200%
[ 7 frame 0,56%
+175%
[ 12 frame 0,96%
+150%
[ 18 frame 1,44%
+125%
[ 35 frame 2,81%
+100%
[ 86 frame 6,90%
+75%
[ 114 frame 9,14%
+50%
[ 127 frame 10,18%
+25%
[ 128 frame 10,26%
--- Average frame size ---
[ 114 frame 9,14%
-25%
[ 188 frame 15,08%
-50%
[ 315 frame 25,26%
-75%
[ 43 frame 3,45%
-100%

high-25, low-25 or what?

iago
17th August 2002, 12:39
@biersauf

In your case, I would increase the high curve value _at least_ up to 200 (since 5.85% of the total frames goes up to that range) according to your report file.

iago

iago
17th August 2002, 12:45
@biersauf

+300%
[ 8 frame 0,64%
+200%
[ 73 frame 5,85%
+100%
[ 475 frame 38,09%
--- Average frame size ---
[ 681 frame 54,61%
-100%

i would set high to 100 and low to 100. (?)
My previous post refers to these values you provided before you edited your post (simultaneously with my reply I guess ;))

iago

biersauf
17th August 2002, 12:53
ok thx. so lets have a look at this (i know its the same as before):

Number of frames = 1253
1247 delta frames and 6 key frames (excluded)
Average frame size = 10.621
Max frame size is +221% over the average
Min frame size is -99% under the average

+225%
[ 8 frame 0,64%
+200%
[ 7 frame 0,56%
+175%
[ 12 frame 0,96%
+150%
[ 18 frame 1,44%
+125%
[ 35 frame 2,81%
+100%
[ 86 frame 6,90%
+75%
[ 114 frame 9,14%
+50%
[ 127 frame 10,18%
+25%
[ 128 frame 10,26%
--- Average frame size ---
[ 114 frame 9,14%
-25%
[ 188 frame 15,08%
-50%
[ 315 frame 25,26%
-75%
[ 43 frame 3,45%
-100%



so first, i have to look at 25%, this means 10.26%, right? then i have to look at the average frame size, which is 10.621. then i have to compare it. after that 10.26 is really close to 10.62 so i have to change high to 25 or is there something i do not understand? if the average frame size contains 22.5 for example i will go to 50%, that means 10,26 + 10,18, is this right?

thx for all...and sry for bad english :)

iago
17th August 2002, 13:20
@biersauf

Well, imho that's not the way you should analyze the report file. If we try to put it more clearly, only by using the report file by intervals 100, which would be easier to comment on (perhaps you can use interval 25 for fine tuning later), we have a result of:

Number of frames = 1253
1247 delta frames and 6 key frames (excluded)
Average frame size = 10.621
Max frame size is +221% over the average
Min frame size is -99% under the average

+300%
[ 8 frame 0,64%
+200%
[ 73 frame 5,85%
+100%
[ 475 frame 38,09%
--- Average frame size ---
[ 681 frame 54,61%
-100%

Here, simply, I would set the low distance to 100 (maybe a bit lower), and high distance to 200-220. Average frame size is the reference point in the calculation, and the other percentages refer to the framesize distribution within the specified ranges (+100/+200/+300, etc.) with respect to the average frame size.

regards,
iago

TheUnforgiven
17th August 2002, 14:46
-do the 1st pass and save the debug info then analyse it with perfectXviD.
-open debugview again try the settings u decided in alt. curve and start the second pass and abort it after 1 second
-look at the debugview window u will see information like this in the beginning:

avg scaled framesize:5039
bias bonus:612 bytes
quant:2 threshold at 242 : -95 percent
quant:3 threshold at 454 : -90 percent
quant:4 threshold at 798 : -84 percent
quant:5 threshold at 1387 : -72 percent
quant:6 threshold at 2412 : -52 percent
quant:7 threshold at 4345 : -13 percent
quant:8 threshold at 9542 : 89 percent
quant:9 threshold at 12844 : 154 percent
quant:10 threshold at 16451 : 226 percent

-look into perfectXviD distribution and u should know now how many frames will take some quantizer.
-tweak the alt. curve settings and start the 2nd pass for one second
and look again and compare if these settings r better or not.
-repeat untill u find the best settings for ur movie without wasting time in encoding.

got the point?;)

biersauf
17th August 2002, 16:22
k, thank you very much. but, i've got some questions:

open debugview again try the settings u decided in alt. curve
so i just start with high-200, low-100 and tweak it?

quant:2 threshold at 242 : -95 percent
quant:3 threshold at 454 : -90 percent
quant:4 threshold at 798 : -84 percent
quant:5 threshold at 1387 : -72 percent
quant:6 threshold at 2412 : -52 percent
quant:7 threshold at 4345 : -13 percent
quant:8 threshold at 9542 : 89 percent
quant:9 threshold at 12844 : 154 percent
quant:10 threshold at 16451 : 226 percent
i only have "quant:2 threshold at 1448 : -82 percent", why?

Number of frames = 1253
1247 delta frames and 6 key frames (excluded)
Average frame size = 10.621
Max frame size is +221% over the average
Min frame size is -99% under the average

+300%
[ 8 frame 0,64%
+200%
[ 73 frame 5,85%
+100%
[ 475 frame 38,09%
--- Average frame size ---
[ 681 frame 54,61%
-100%

ok, now i see that i've got 475 frames with quantizer 2 right (quantizer 2 = ~82%)? how should i tweak my settings (should i tweak them to quan. 2???)?

Didée
17th August 2002, 18:08
To me, one thing is not clear:

If low distance is set to e.g. 80% - what happens to the frames in the range of 81%-100% distance?

Will they get quant2 ? (would seem logical to me)

If I remember right, Out_pinged mentioned in the long AltCC thread that those frames would get "AverageQuant".

This is not trivial. Imagine an encoding with AvQuant ~ 7-8. If those tiny frames get quant2, quality is safe.
If they get AvQuant, there is the risk they get blocky - and trail through up to the next keyframe.

So, what does happen to frames beyond (below) of LowDistance?

Vash
18th August 2002, 18:14
@TheUnforgiven: This might very well be off topic, but how about a counter indicating how many changes in quantizers occour with certain settings? If quantizers change too often picture quality seems to go down the drain, at least for me
;)

Aktan
19th August 2002, 03:35
@Vash

This tool is for 1-pass, in which all quants are 2 :) .

rui
19th August 2002, 09:04
Ok, i did a test with the movie "The Replacements" and this is what i got:
1st pass analise:

Number of frames = 170202
167365 delta frames and 2837 key frames (excluded)
Average frame size = 13.176
Max frame size is +627% over the average
Min frame size is -99% under the average

+650%
[ 1 frame 0,00%
+625%
[ 0 frame 0,00%
+600%
[ 1 frame 0,00%
+575%
[ 6 frame 0,00%
+550%
[ 12 frame 0,01%
+525%
[ 13 frame 0,01%
+500%
[ 6 frame 0,00%
+475%
[ 4 frame 0,00%
+450%
[ 5 frame 0,00%
+425%
[ 5 frame 0,00%
+400%
[ 8 frame 0,00%
+375%
[ 12 frame 0,01%
+350%
[ 41 frame 0,02%
+325%
[ 122 frame 0,07%
+300%
[ 282 frame 0,17%
+275%
[ 329 frame 0,20%
+250%
[ 391 frame 0,23%
+225%
[ 762 frame 0,46%
+200%
[ 1490 frame 0,89%
+175%
[ 2367 frame 1,41%
+150%
[ 3615 frame 2,16%
+125%
[ 5260 frame 3,14%
+100%
[ 6789 frame 4,06%
+75%
[ 10468 frame 6,25%
+50%
[ 13716 frame 8,20%
+25%
[ 18149 frame 10,84%
--- Average frame size ---
[ 25681 frame 15,34%
-25%
[ 30754 frame 18,38%
-50%
[ 30554 frame 18,26%
-75%
[ 10004 frame 5,98%
-100%

So, i tried the following setings in alt cc.: high agression; high-350%; low-25%

I got this report from Debugview analyzer, from MoonWalker:
DebugView analyzer for XviD codec v0.10 by MoonWalker
e-mail : s_ilias@gmx.net


Quantizers Analisis
---------------------

Quantizers Used For Movie :
------------------------------
Quant 2 Used : 161 Times, Percentage Used : 0.10%
Quant 3 Used : 7612 Times, Percentage Used : 4.68%
Quant 4 Used : 36189 Times, Percentage Used : 22.23%
Quant 5 Used : 40209 Times, Percentage Used : 24.70%
Quant 6 Used : 28885 Times, Percentage Used : 17.74%
Quant 7 Used : 24611 Times, Percentage Used : 15.12%
Quant 8 Used : 13744 Times, Percentage Used : 8.44%
Quant 9 Used : 7249 Times, Percentage Used : 4.45%
Quant 10 Used : 2733 Times, Percentage Used : 1.68%
Quant 11 Used : 1084 Times, Percentage Used : 0.67%
Quant 12 Used : 268 Times, Percentage Used : 0.16%
Quant 13 Used : 59 Times, Percentage Used : 0.04%
Quant 14 Used : 13 Times, Percentage Used : 0.01%

Average Quantizer Used for Movie : 5.731

Quantizers Used For Credits :
--------------------------------
Quant 20 Used : 7385 Times.

MPEG Quantization Type Used 170202 timed, Percentage Used : 100.00%

Quantizers prevented from rising too steeply 0 times


Intra-Frame (Key-Frame) Quantizers
------------------------------------

Movie
-------
Quant 5 Used : 2750 Times, Percentage Used : 96.93%
Quant 6 Used : 12 Times, Percentage Used : 0.42%
Quant 7 Used : 50 Times, Percentage Used : 1.76%

Credits
---------
Quant 20 Used : 25 Times, Percentage Used : 0.88%

Number Of Consecutive I-Frames : 50

Inter-Frame (P-Frame) Quantizers
------------------------------------

Movie
-------
Quant 2 Used : 161 Times, Percentage Used : 0.10%
Quant 3 Used : 7612 Times, Percentage Used : 4.55%
Quant 4 Used : 36189 Times, Percentage Used : 21.62%
Quant 5 Used : 37459 Times, Percentage Used : 22.38%
Quant 6 Used : 28873 Times, Percentage Used : 17.25%
Quant 7 Used : 24561 Times, Percentage Used : 14.68%
Quant 8 Used : 13744 Times, Percentage Used : 8.21%
Quant 9 Used : 7249 Times, Percentage Used : 4.33%
Quant 10 Used : 2733 Times, Percentage Used : 1.63%
Quant 11 Used : 1084 Times, Percentage Used : 0.65%
Quant 12 Used : 268 Times, Percentage Used : 0.16%
Quant 13 Used : 59 Times, Percentage Used : 0.04%
Quant 14 Used : 13 Times, Percentage Used : 0.01%

Credits
---------
Quant 20 Used : 7360 Times, Percentage Used : 4.40%



Frame Analisis
----------------

Number Of Intra-Frames (Key-Frames) : 2837
Number Of Inter-Frames (P-Frames) : 167365
Total Number Of Frames : 170202

1.67% of the Movie is Intra-Frames (Key-Frames)
98.33% of the Movie is Inter-Frames (P-Frames)



Size Analysis
----------------

1-Pass Size : -1990093946 Bytes or 2250852 KBytes
Scaled Size : 732106400 Bytes or 714947 KBytes
Actual Size : 732105724 Bytes or 714946 KBytes

Usefull Statistics
------------------
Compressibility : -36.79%
Relative Quality of XviD avi : 34.90%
Absolute Quality of XviD avi : 88.81%

I must try now with default cc, to see if my choosen alt. cc. parameters were anyb good. But this is the time that i got the resulting movie to look best, when using 1 cd encode for this movie :)

milan
19th August 2002, 09:33
Hi,

I'm currently rewriting ffvfw (if you don't know it is something as a counterpart to ffdshow for encoding) and I think it could be useful to intergrate PerfectXviD to ffvfw. Honestly I haven't tried this tool yet, but after reading this thread I think it is possible. All I need is to try PerfectXviD and see how it realy works and then obtain its source code.

Milan

rui
19th August 2002, 11:20
Ok, i am getting confused here :D

I made some tests using “The Replacements” trailer, and using the latest Koepi's build.

Using the above mentioned post descriving the method of stoping the 2nd pass and looking at the quantizer distribution, i tweaked a little the config i had obtained from XvidPerfect. In my original alt cc config i had chosen high-275% low-25% and high agression. By changing and experimenting with the values, i got the lower fluctuaction in quantizer values shown in the begining of the 2nd pass by choosing high-275% low-50% with high agression. (is this the right way of doing things?):confused:

I tried 3 diferent settings:

High agression; high 275; low 50 -> average quantizer 7,344, but used quantizers till 16

Medium agression; high 275; low 50 -> average quantizer 7,354 but used quantizers till 15

Koepi’s default -> average quantizer 7,404 but it used max quantizer value of 14

So, i am still in doubt: it’s best to get lower max quantizers, or lower average quantizer?:confused:
Visually, the encodes a very similar between them.

EDIT:By TheUnforgiven
i think this depend on what frames u want to optimize high,low or something in between.

Humm..By this you mean that if i want to optmize, lets say, low, i should choose lower max quantizer?

sierrafoxtrot
19th August 2002, 11:50
aaaaaaaaaa! i'm confused too rui :D but i think average quantizers don't mean a whole lot, because it's the quantizer distribution for framesize that makes the most difference. if theunforgiven could actually quant used vs 1-pass framesize, we might be on to something.

i've done encodes of rumble in the bronx and the last castle over the weekend using koepi's 17-08-2002 build, with Mod/H293 Hi/Lo 100/500 and 500/100 with 0 or 30 strength and medium and low aggression. to be honest, i couldn't tell the difference between the majority of the encodes anymore. although i did find that using h263 low aggression hi/lo 250/50, 30 strength, 20% i-frame boost gave the best overall encodes.

@rui looking at the framesize distribution for your replacements 1st-pass, you can see the the framesizes are pretty well spread. i guess that means i would choose high aggression, and hi/lo 350/50, which is almost identical to what you're doing.

after spending so long on this, i wonder if the best solution isn't to just do a vanilla 2nd pass, identify the scenes which are crap, and then manually scale the curve using gKnot using the selection and push for a higher bitrate in those sections. imho, alt cc seems to not make a tremendous difference. :confused:

rui
19th August 2002, 12:32
Originally posted by sierrafoxtrot
imho, alt cc seems to not make a tremendous difference. :confused:

That was my conclusion some time ago, so i started using default cc. The diferences were simply too small.

But i really have good hopes for this new tool that TheUnforgiven has given us. It's just a matter of getting to some definitive way of doing things right. But i have to admit that, at least i am still a long way of getting there :rolleyes:

LigH
19th August 2002, 13:55
I have absolutely no clue what this program is useful for; at least in the current version (0.3), it is not able to handle my output, and always crashes:

I'm using DebugView 4.13 (W98) and August builds of XviD. They are producing a different log file format compared to the shipped example file. To get into detail, the second column contains a timecode - this is a floating point number in the example, but the clock time in my log file created a few minutes ago; the third column starts with text in brackets; in the example file, this is "[340]", in my current log it is "[Virtuald]".

After reading the current log, it immediately crashes with a "Runtime error '5': Invalid function call or invalid argument" (I guess it is not able to read the clock time values); after accidently reading a DivX5 log file, it crashes with "Runtime error '6': Overflow" (you see - absolutely no error tolerance).

Reading the example log file works fine; but I don't understand what it wants to tell me: The average frame size is between -100% and +100%, great; but what shall I enter now, 20% or 70% maybe?!?

TheUnforgiven
19th August 2002, 15:43
@rui and sierrafoxtrot
dont give up easily.i sent a PM to koepi to know his opinion about the tool and how to decide things from it also see my reply to bersauf below and tell me ur opinion about this basic method.
@LigH
u should open the log file for the first pass only.
@milan
welcome man.
unfortunately the source is in VB not C but the idea is very simple and can be ported to c easily the best way i think is to throw this analysis in the debug print. what do u think?
@didee
i seems logical to me that they get a quantizer of 2 but i dont know for sure. ask one of the developers or wait until i do a test for u. i'll modify the tool to give the frame number of the smallest frame and then i'll see what quantizer the codec gives to it.
@unplugged
i agree with u about the need for the developers read @rui above
@biersauf
here is an easy way which gives good but not necessarily the best result:
set low to the minimum percentage.
set high to somewhere where most of the frames r below ignoring all the very big few frames.
if most of the frames r around the average set aggression to low
if the frames distribute equally choose medium
if there r many frames far from the average choose high aggression.
@all what do u think of this method just above.

Easy123
19th August 2002, 16:48
@TheUnforgiven

Well, I´d like to say thanks for your great Tool...

In the last two Days, I did two movies with it. The Latter was "Rambo- First Blood" (PAL 25fps) which I finisched only ten Minutes ago... With the help of your Tool I get more quality out of Movies...

Here is what I got with your Tool:



Number of frames = 133962
132693 delta frames and 1269 key frames (excluded)
Average frame size = 5.938
Max frame size is +828% over the average
Min frame size is -99% under the average

+850%
[ 1 frame 0,00%
+825%
[ 0 frame 0,00%
+800%
[ 0 frame 0,00%
+775%
[ 0 frame 0,00%
+750%
[ 0 frame 0,00%
+725%
[ 3 frame 0,00%
+700%
[ 2 frame 0,00%
+675%
[ 2 frame 0,00%
+650%
[ 3 frame 0,00%
+625%
[ 1 frame 0,00%
+600%
[ 3 frame 0,00%
+575%
[ 3 frame 0,00%
+550%
[ 4 frame 0,00%
+525%
[ 4 frame 0,00%
+500%
[ 9 frame 0,01%
+475%
[ 13 frame 0,01%
+450%
[ 14 frame 0,01%
+425%
[ 13 frame 0,01%
+400%
[ 27 frame 0,02%
+375%
[ 26 frame 0,02%
+350%
[ 60 frame 0,05%
+325%
[ 122 frame 0,09%
+300%
[ 302 frame 0,23%
+275%
[ 507 frame 0,38%
+250%
[ 824 frame 0,62%
+225%
[ 1113 frame 0,84%
+200%
[ 1355 frame 1,02%
+175%
[ 1572 frame 1,18%
+150%
[ 2085 frame 1,57%
+125%
[ 3104 frame 2,34%
+100%
[ 4880 frame 3,68%
+75%
[ 7151 frame 5,39%
+50%
[ 10070 frame 7,59%
+25%
[ 14811 frame 11,16%
--- Average frame size ---
[ 21295 frame 16,05%
-25%
[ 26818 frame 20,21%
-50%
[ 27010 frame 20,36%
-75%
[ 4122 frame 3,11%
-100%

So I set Alt.CC settings to the Following:

Curve Agresseion: Low (I got best resultes with it)
High: 725% and Low: 95%

And here´s the result of dbg.exe:
Quantizers Analisis
---------------------

Quantizers Used For Movie :
------------------------------
Quant 2 Used : 63448 Times, Percentage Used : 47.36%
Quant 3 Used : 70445 Times, Percentage Used : 52.59%
Quant 4 Used : 69 Times, Percentage Used : 0.05%

Average Quantizer Used for Movie : 2.527

No credits encoding!!

MPEG Quantization Type Used 133893 timed, Percentage Used : 99.95%

H.263 Quantization Type Used 69 timed, Percentage Used : 0.05%

Quantizers prevented from rising too steeply 0 times


Intra-Frame (Key-Frame) Quantizers
------------------------------------

Movie
-------
Quant 2 Used : 1147 Times, Percentage Used : 91.25%
Quant 3 Used : 110 Times, Percentage Used : 8.75%

Credits
---------

Number Of Consecutive I-Frames : 108

Inter-Frame (P-Frame) Quantizers
------------------------------------

Movie
-------
Quant 2 Used : 62301 Times, Percentage Used : 46.95%
Quant 3 Used : 70335 Times, Percentage Used : 53.00%
Quant 4 Used : 69 Times, Percentage Used : 0.05%

Credits
---------



Frame Analisis
----------------

Number Of Intra-Frames (Key-Frames) : 1257
Number Of Inter-Frames (P-Frames) : 132705
Total Number Of Frames : 133962

0.94% of the Movie is Intra-Frames (Key-Frames)
99.06% of the Movie is Inter-Frames (P-Frames)



Size Analysis
----------------

1-Pass Size : 816995837 Bytes or 797847 KBytes
Scaled Size : 641222688 Bytes or 626194 KBytes
Actual Size : 641225597 Bytes or 626196 KBytes

Usefull Statistics
------------------
Compressibility : 78.49%
Relative Quality of XviD avi : 79.15%
Absolute Quality of XviD avi : 98.42%

And I can´t tell the Difference from the DVD when watched on my TV...

So thx Dude...

Easy123

rui
19th August 2002, 17:23
Originally posted by TheUnforgiven
@rui and sierrafoxtrot
dont give up easily.i sent a PM to koepi to know his opinion about the tool and how to decide things from it also see my reply to bersauf below and tell me ur opinion about this basic method.

TheUnforgiven, i still didn’t give up ;)

Like i said in my post, i tried to tweak the values that your tool gave me using your rmethod. I changed from 275-25 to 275-50, because this way i got less fluctuaction in quantizers in the start of the 2nd pass.
But, using you method i got a report in the begining of the 2nd pass saying that it would use only quantizers till 12 at a percentage of 196%. But in the end it used quantizers till 14 or more, depending on the configuration.

Probably i am interpreting the results i got wrong (what those values in the small report in the begining of the 2nd pass mean? That those will be the quantizers used? At least it was what i assumed)

Again, i confess that i am a little lost, but i will try this latest tip you gave us. Probably will only post my experiences tomorrow because i am having some problems on the internet at home.

Aktan
19th August 2002, 18:37
@TheUnforgiven

Great tool, but I notcie all this information from dbgview is all there in the .stats file already. Wouldn't it be easier to analize the .stats file instead of making another log?

@all
Since you use dbgciew analizer for 2-pass and know how many frames are used in which quants, you can also pinpoint the exact frame where the highest quant was used. Then you can just watch that area and see if you would notice blocks! It would then answer if lower avg quant or lower max used quant is better :)

LigH
19th August 2002, 19:05
Originally posted by TheUnforgiven
@LigH
u should open the log file for the first pass only.


This log file was indeed created with a 1st-pass run! I'll list a few lines here:

Example file:
00000000 0.00000000 [340] 1st-pass: size:62583 total-kbytes:61 intra quant:2 H.263 kblocks:680 mblocks:0
00000001 0.08030015 [340] 1st-pass: size:10279 total-kbytes:71 inter quant:2 H.263 kblocks:0 mblocks:607
00000002 0.23377245 [340] 1st-pass: size:9901 total-kbytes:80 inter quant:2 H.263 kblocks:0 mblocks:608

My new file:
00000000 14:20:39.345 [Virtuald] 1st-pass: size:1614 total-kbytes:1 intra quant:2 H.263 kblocks:576 mblocks:0
00000001 14:20:39.440 [Virtuald] 1st-pass: size:83 total-kbytes:1 inter quant:2 H.263 kblocks:0 mblocks:0
00000002 14:20:39.530 [Virtuald] 1st-pass: size:83 total-kbytes:1 inter quant:2 H.263 kblocks:0 mblocks:0

See the differences?

MoonWalker
19th August 2002, 19:58
Originally posted by Aktan
@TheUnforgiven

Great tool, but I notcie all this information from dbgview is all there in the .stats file already. Wouldn't it be easier to analize the .stats file instead of making another log?




Well, I am currently trying to implement this in my dbgview analyzer prog...

Originally posted by Aktan

@all
Since you use dbgciew analizer for 2-pass and know how many frames are used in which quants, you can also pinpoint the exact frame where the highest quant was used. Then you can just watch that area and see if you would notice blocks! It would then answer if lower avg quant or lower max used quant is better :)

It's easy..I can print at the graph.txt file the higher quants(2 or 3)..Is this what are you talking about??

MoonWalker

Aktan
19th August 2002, 23:11
Originally posted by MoonWalker


It's easy..I can print at the graph.txt file the higher quants(2 or 3)..Is this what are you talking about??



That be great cause right now i search for the highest quant in dbg.log then search for the same stats1 size, scaled size and actual size in graph.txt to find the frame. If you could please print all quants not just quants aboive 3 :) . Tho I forgot to mention two bugs I noticed. 1. It seems the "Quantizers prevented from rising too steeply # times" part of your analizer broke. 2. The frames in graph.txt are 1 frame off. For example frame 601 is really frame 600 in VDub, aka frames start at frame 0, not 1 :) .

iago
20th August 2002, 03:35
@MoonWalker
Originally posted by Aktan

It seems the "Quantizers prevented from rising too steeply # times" part of your analizer broke. Maybe a bit off-topic, but yes, v0.10 doesn't report how many times the quantizers were about to rise too steeply :).

regards,
iago

rui
20th August 2002, 09:25
Well, after some testing last night, I came to the following conclusion:

For those that don’t want to experiment a lot to find that good config in alt cc., use Koepi’s default setting. In all my experiments, it gets good (low) average quantizer using low quantizers all around.

That’s it.

But for those that like to play a little, you can use TheUnexpected tool to get more refined values to put in high and low. Also I have been using his advices when choosing low, medium or high aggression, but medium aggression is almost always a go. I for one could most of the times improve a little bit (sometimes only really very little bit) by changing the values Koepi has for default to ones chosen by observation of the frame size distribution.

For instance, if the report looks like this:

Number of frames = 3570
3440 delta frames and 130 key frames (excluded)
Average frame size = 17.478
Max frame size is +270% over the average
Min frame size is -99% under the average

+275%
[ 2 frame 0,06%
+250%
[ 2 frame 0,06%
+225%
[ 13 frame 0,38%
+200%
[ 23 frame 0,67%
+175%
[ 50 frame 1,45%
+150%
[ 44 frame 1,28%
+125%
[ 58 frame 1,69%
+100%
[ 168 frame 4,88%
+75%
[ 132 frame 3,84%
+50%
[ 300 frame 8,72%
+25%
[ 546 frame 15,87%
--- Average frame size ---
[ 858 frame 24,94%
-25%
[ 697 frame 20,26%
-50%
[ 201 frame 5,84%
-75%
[ 209 frame 6,08%
-100%

Then I choose, for high the value 275, because at that value I still have some 2 frames, and in my test (few frames) they are significant. For a full movie, maybe they aren’t, and one could choose a lower value. For low, since I have 209 frames at –100%, I have chosen 100%. If anybody thinks that this is wrong, fell free to correct me.

Also, I found out that, at least in my experiments, the strength value given by Koepi is the best (30%). Of course that there can be cases where a different value could be better, but this one always gave me the best results.

That’s it, hope someone finds this helpful.

TheUnforgiven
20th August 2002, 15:25
@easy123
i am glad u found it useful.
@rui
thanx again for ur useful test. i noticed also in my tests that the difference in the average quantizer is very small but the good thing is that it is on perfectxvid side. and dont forget that we can also optimize big or small frames even if the average quantizer is not changed.
you can use TheUnexpected tool to get more refined values to put in high and low
i think the alt cc is the unexpected ,unless some expert answers our questions, not the tool :D :D :D
@aktan
Great tool, but I notcie all this information from dbgview is all there in the .stats file already. Wouldn't it be easier to analize the .stats file instead of making another log?
yes i know but the stat file is not a text file and i don't know how to read info from it. if u (or anyone)have info regarding this please post it on the thread of MoonWalker about the stat file
http://forum.doom9.org/showthread.php?threadid=31762
@LigH
for the tool both of those r no problem cuz i read the size and the typeof the frame without the other shit.
do u still have problems even with the first pass only log file?
if so just cut some lines (10 is enough) and save it in another LOG file and try. if still there is a problem send that file to my mail.
@MoonWalker
welcome man. i use ur great tool but would u please not request the log file to have "dbg.log" name and just take any file as an argument. i need this cuz i often saves many log files in the same directory.
please let me know if u discovered how to read the stat file.

rui
20th August 2002, 16:52
Originally posted by TheUnforgiven
i think the alt cc is the unexpected ,unless some expert answers our questions, not the tool :D :D :D

Sorry. I meant TheUnforgiven tool :p

MoonWalker
21st August 2002, 09:00
Originally posted by TheUnforgiven
@easy123
@MoonWalker
welcome man. i use ur great tool but would u please not request the log file to have "dbg.log" name and just take any file as an argument. i need this cuz i often saves many log files in the same directory.
please let me know if u discovered how to read the stat file.

Hi :)..Well I will change this..I had it this way cause it very very simple to just double click the .exe and run..But I will do it to take any file as argument,along with some bug fixes..

MoonWalker


BTW Which languane do you programm you prog??

crOOk
21st August 2002, 13:16
Your tool is great. I've done 7 encodes with it and they were all beyond my expectations (one of them looked alright with a 40% ratio). It is very easy to use and I don't need to configure anything besides clipping the credits.
It would be great if I could choose steps of 10% by default. Even 1% steps would be a help for getting the Low Value. 10% already works but when I enter lower values it crashes. Thanx a lot!

TheUnforgiven
21st August 2002, 17:32
@rui
i know, just joking ;D
@moonWalker
-good news waiting for that.
-it is in VB.
@crOOk
-i am happy u found it useful.
-i'll do that in future versions.
@all
what r ur suggestions to enhance the tool?
graph? more stats?

Sigmatador
21st August 2002, 20:18
Can PerfectXvid calculate a suggest value for low/high ?

MoonWalker
21st August 2002, 23:05
Originally posted by iago
@MoonWalker
Maybe a bit off-topic, but yes, v0.10 doesn't report how many times the quantizers were about to rise too steeply :).

regards,
iago

Hmm..Maybe the struct of the log file has changed..Has anyone has a log file with some "prevention" on it? If anyone has please e-mail or PM me..

Thanks,
MoonWalker

iago
22nd August 2002, 01:35
@MoonWalker

I tried two times to send you the e-mail, but I got "undeliverable" notification from the mail server each time :(. Maybe you can provide another e-mail address, so that I can send the sample log to you.

regards,
iago

MoonWalker
22nd August 2002, 01:40
Originally posted by iago
@MoonWalker

I tried two times to send you the e-mail, but I got "undeliverable" notification from the mail server each time :(. Maybe you can provide another e-mail address, so that I can send the sample log to you.

regards,
iago

It's ok iago..I tried and I produced a "prevention"...I fixed it but it's a quick hack..I will realease it in a couple of minutes..

Thanks anyways :)
MoonWalker

iago
22nd August 2002, 01:52
OK, nice to hear that good news...
:)
iago

MoonWalker
22nd August 2002, 02:01
v0.11 released

Please post any bugs and/or suggestions in this thread : http://forum.doom9.org/showthread.php?s=&threadid=31970

MoonWalker

P.S
@TheUnforgiven
Sorry for being a little off-topic at your thread :)

TheUnforgiven
22nd August 2002, 07:29
@Sigmatador
u can read my suggestions in my previous posts but since i am not sure if it is the best way i will not put it in the tool.

@MoonWalker
no problem my friend i am interested in ur tool too.

TheUnforgiven
22nd August 2002, 07:36
ver0.35 changes

-now it will take half the time to analyse a file.
-shows number of frames with size equals to the average.
-improved error control when opening files not in the proper format.
-displays the time taken to analyse a file.
-new interval lengths, 10.

u can get it from there:
http://www.everwicked.com/forums/showthread.php?s=&postid=8248#post8248

LigH
22nd August 2002, 07:37
@TheUnforgiven:

Yes, I still have problems with my log file, even if I cut it down to just a few lines. So please send me a mail that I can reply back to you, because I cannot attach the log file when writing you right out of the doom9 forum.

Or shall I attach it openly here? I think even the 3 lines I recently posted shall be almost enough.
____

Wait a minute - you were posting version 0.35 just at the same moment I was writing this. I'll check again...
____

No, version 0.35 still crashes with "Runtime error '5'" - I'll definitely have to send you my log file.

milan
22nd August 2002, 07:49
Originally posted by TheUnforgiven
-now it will take half the time to analyse a file.


It realy proved to be useful reading first pass stats file instead of Debugview log. Yesterday I implemented PerfectXviD features into ffvfw and stats files are analyzed almost instantly. I don't know how Visual Basic handles binary reading and custom structs, but you should definitely try it.

TheUnforgiven
22nd August 2002, 08:40
@LigH
ok send it to unforgiven@soon.com.
and i'll try the lines u post as soon as possible.
@milan
i agree with u, the stat file would be better. but i dont know how to read it untill now.
this thread gives the struct
http://forum.doom9.org/showthread.php?s=&threadid=31762
but could u please give me some hints.

milan
22nd August 2002, 08:54
The struct in DaveEL's reply is well described. For your calculations you need bytes value (first four bytes of frame record) and to know if this is keyframe of not. This is stored in quant field (integer starting at 44th byte of frame record). When this field is in range 1-31 it is P frame, when it is bigger (something like 2147483650) it is I frame.
Before analyzing frames you must read first four bytes of stats file, where stats version is stored (must be -20).

OUTPinged_
22nd August 2002, 16:22
Well i tried to come up with something usefull, but it have turned out there is not good all-around answer.


The fun part is that finding an easy high/ low pass value isnt hard. Honest. Drop 10% of largest and 10% of smallest frames and it will be ok. ( I mean your rip will be screwed not because of distance :-)

But setting good high/low dist. and crappy mrq/aggresssion wont make a bad rip look good.

I personally would not recommend using "strength" setting because it doesnt make life easyer and it has to be recalculated for each rip to not screw things up.

For min_rel_qual, you have to know an average quantizer and your desired max quantizer.



So, here we go:

For our pretty curve compression to work flawlessly, we want mrq set so that at it's highest the quantizer value will reach max and go no futher. Smart guys did already read my explainings of alt.cc (hint hint) and that part is easy.

To be sure, set a quantizer cap :-)

Setting a low/medium compression depends if you want to have a more constant distribtuion of quantizers or not. for stuff with loads of action low agg will work better, use med agg otherwise (or if you have a crappy source to start with).

Here comes our little distance settings.

First, dont forget that xvid suffers same problems with very dark scenes as all mpeg4 codecs do. For rips with no dark scenes use low dist that drops about 20-30% of smallest frames. Rips with excessive night scenes should benefit from low dist. close to 100. Luma masking should be off in that case, too.

High distance value depends on how big quantizer range you have between average quantizer and max quantizer. Decide how much frames you want to have the highest quantizer value. If a range is 4 (ie av=4 max=8) then you can cut 25% of largest frames. If it is 10 (av=6,max=16), cut 10% of frames. Pretty simple, ne?

I know that all above is extremely stupid idea, and all suggestions are welcome of course.

crOOk
22nd August 2002, 17:31
I'd like to be able to specify a start frame for the credits so I don't need to cut them off manually... That would be great, thanx.

Sigmatador
22nd August 2002, 21:46
when i obtain this:

Number of frames = 12496
132934 delta frames and 1177 key frames (excluded)
Average frame size = 5.367
Max frame size is +734% over the average
Min frame size is -98% under the average

i have to set low at 98 and high at 734 right ?

OUTPinged_
22nd August 2002, 22:57
wrong. read the stuff 2 posts above :/

Sigmatador
22nd August 2002, 23:46
ok it's wrong but there 's 2 or 3 posts where this setting have been used to obtain good quality :confused: .

so sorry but your post isn't easy to understand. the value obtained with your method are different than ones i have read above in this thread (from the right perfectxvid log ^^). so i did a mistake but where ? can you give me a clear and simply example from a perfectxvid log to help me understanding plz ?

ciao all ^^

OUTPinged_
23rd August 2002, 11:21
ok it's wrong but there 's 2 or 3 posts where this setting have been used to obtain good quality

Then you are welcome to try it and post your impression here.

Wimpy
23rd August 2002, 16:05
Looks like a really intersting tool, great work! :-) I would just like to second the request for it to generate it's reports from the 1st pass .stats file rather than the debugview .log file.

Any progress on this?

I am still new to XviD and trying to learning more about it, but is there anything in the .stats files which could be used to estimate the amount of motion detected in a given video clip? It would be useful to report this if possible to help guide a decision for Curve Aggression settings - Low, Medium or High.

Good luck with future developments :-D

iago
23rd August 2002, 21:06
@OUTPinged_ and everybody

Personally I prefer to use Koepi's latest with default AltCC parameters, but to make things about this tool clearer for everyone (and for me too), what about taking a first pass reports file (together with other useful encoding data) on a well known movie and share our comments and opinions on it (including all AltCC parameters, quantizer restrictions, etc.)?

For example:
------------
Movie: Matrix/region2/PAL 16:9
2hr-11min
640*272/SimpleResize
aimed video size: 656500kb (~641mb)

uManiac's build (12.08.2002)
First pass:
6.UltraHigh
H.263
No lumi
------------

I'll post the first pass results and the PerfectXviD reports file when the first pass is finished.

best regards,
iago

Sigmatador
23rd August 2002, 21:56
@OUTPinged_
i try i try ^^ (going to post ) but i prefer testing with best values ^^

iago
24th August 2002, 04:09
MATRIX First Pass Results:
(with the above mentioned settings)

1stPass/2ndPass ratio: 1220/641: 1.90
GKnot Compressibility: 55.9

PerfectXviD Report:

Number of frames = 196155
193321 delta frames and 2834 key frames (excluded)
Average frame size = 6.374
Max frame size is +1.213% over the average
Min frame size is -98% under the average

+1250%
[ 1 frame 0,00%
+1200%
[ 2 frame 0,00%
+1150%
[ 2 frame 0,00%
+1100%
[ 7 frame 0,00%
+1050%
[ 3 frame 0,00%
+1000%
[ 32 frame 0,02%
+950%
[ 63 frame 0,03%
+900%
[ 98 frame 0,05%
+850%
[ 59 frame 0,03%
+800%
[ 98 frame 0,05%
+750%
[ 118 frame 0,06%
+700%
[ 91 frame 0,05%
+650%
[ 86 frame 0,04%
+600%
[ 147 frame 0,08%
+550%
[ 181 frame 0,09%
+500%
[ 317 frame 0,16%
+450%
[ 482 frame 0,25%
+400%
[ 543 frame 0,28%
+350%
[ 689 frame 0,36%
+300%
[ 1021 frame 0,53%
+250%
[ 1861 frame 0,96%
+200%
[ 3778 frame 1,95%
+150%
[ 7351 frame 3,80%
+100%
[ 14706 frame 7,61%
+50%
[ 31657 frame 16,38%
--- Average frame size --- { 970 frame 0,50%
[ 82764 frame 42,81%
-50%
[ 46194 frame 23,89%
-100%

Waiting for your comments. Best wishes.
iago

cooper99
24th August 2002, 09:13
I am trying to figure out how to calculate minimum relative quality. In the advanced settings for 2nd-pass, when I disable Automatic MRQ, strength is greyed out. Now, if I was to use the formula 100-(100-EncodeQuality)*strength=MRQ, where do I come up with the variable, strength?
After having made the 1st pass, it resulted in a stats file. Can I get the EncodeQuality(compressibility) in Gordian Knot using this stats file?

@OUTPinged_

You mentioned planning the Quantizer Cap. I seem to know my avg. and maximum quant. from the following analysis:

**************************************************************
Quantizers Analisis
---------------------

Quantizers Used For Movie :
------------------------------
Quant 2 Used : 305 Times, Percentage Used : 0.18%
Quant 3 Used : 21722 Times, Percentage Used : 12.70%
Quant 4 Used : 94001 Times, Percentage Used : 54.95%
Quant 5 Used : 45759 Times, Percentage Used : 26.75%
Quant 6 Used : 7642 Times, Percentage Used : 4.47%
Quant 7 Used : 1487 Times, Percentage Used : 0.87%
Quant 8 Used : 149 Times, Percentage Used : 0.09%
Quant 9 Used : 15 Times, Percentage Used : 0.01%
Quant 10 Used : 1 Times, Percentage Used : 0.00%

Average Quantizer Used for Movie : 4.256

Intra-Frame (Key-Frame) Quantizers
------------------------------------

Movie
-------
Quant 4 Used : 1575 Times, Percentage Used : 67.65%
Quant 5 Used : 87 Times, Percentage Used : 3.74%
Quant 6 Used : 136 Times, Percentage Used : 5.84%
Quant 7 Used : 475 Times, Percentage Used : 20.40%

Number Of Consecutive I-Frames : 471

Inter-Frame (P-Frame) Quantizers
------------------------------------

Movie
-------
Quant 2 Used : 305 Times, Percentage Used : 0.16%
Quant 3 Used : 21722 Times, Percentage Used : 11.73%
Quant 4 Used : 92426 Times, Percentage Used : 49.90%
Quant 5 Used : 45672 Times, Percentage Used : 24.66%
Quant 6 Used : 7506 Times, Percentage Used : 4.05%
Quant 7 Used : 1012 Times, Percentage Used : 0.55%
Quant 8 Used : 149 Times, Percentage Used : 0.08%
Quant 9 Used : 15 Times, Percentage Used : 0.01%
Quant 10 Used : 1 Times, Percentage Used : 0.00%
***************************************************************

Let's quote from your "Alt-cc options explained" thread:

************************************************************
To make things more simple: if 1/2pass ratio is 66%(av. quant.=3) and mrq=50, your maximum quantizer will not rise higher than quant6, even on _very_ intense scene.
************************************************************

I'm quite confused on how you relate 1/2pass ratio and MRQ with quant6. Can you explain a little bit about it? BTW, I might be wrong in understanding 1/2pass ratio.


iago

from your Matrix analysis:

******************************
1stPass/2ndPass ratio: 1220/641: 1.90
GKnot Compressibility: 55.9
******************************

Would Compressibility be used to figure MRQ? Correct me if I'm wrong.

THX for any feedback. I hate that I'm a newbie. I'm trying.

cooper

iago
24th August 2002, 15:05
MATRIX First Pass Results:
(with the above mentioned settings)

1stPass/2ndPass ratio: 1212/641: 1.89

PerfectXviD Report:

Number of frames = 196155
193419 delta frames and 2736 key frames (excluded)
Average frame size = 6.335
Max frame size is +1.215% over the average
Min frame size is -98% under the average

+1250%
[ 1 frame 0,00%
+1200%
[ 2 frame 0,00%
+1150%
[ 5 frame 0,00%
+1100%
[ 4 frame 0,00%
+1050%
[ 3 frame 0,00%
+1000%
[ 25 frame 0,01%
+950%
[ 32 frame 0,02%
+900%
[ 55 frame 0,03%
+850%
[ 83 frame 0,04%
+800%
[ 97 frame 0,05%
+750%
[ 90 frame 0,05%
+700%
[ 109 frame 0,06%
+650%
[ 120 frame 0,06%
+600%
[ 172 frame 0,09%
+550%
[ 158 frame 0,08%
+500%
[ 332 frame 0,17%
+450%
[ 521 frame 0,27%
+400%
[ 524 frame 0,27%
+350%
[ 690 frame 0,36%
+300%
[ 1084 frame 0,56%
+250%
[ 1916 frame 0,99%
+200%
[ 3833 frame 1,98%
+150%
[ 7340 frame 3,79%
+100%
[ 14688 frame 7,59%
+50%
[ 31477 frame 16,27%
--- Average frame size --- { 1986 frame 1,03%
[ 82618 frame 42,71%
-50%
[ 46447 frame 24,01%
-100%
-------------------------------------------------

PS: I thought it could also be useful to compare the first pass results with two different builds.

best regards,
iago

TheUnforgiven
25th August 2002, 15:31
i am trying the stat file thing now..

@LigH
the solution for ur problem is in ur mail now.

TheUnforgiven
26th August 2002, 07:44
ver 0.5 changes:
-now process the stats file no need for the debug log file.
-instant results (lightning speed)
-calculates bits/(pixel*frame) of the first pass.

get it from here:
http://www.everwicked.com/forums/showthread.php?s=&postid=8333#post8333

LigH
26th August 2002, 08:23
Oh wow - just that you created a working version for my DebugView format, I don't need it anymore :D - But reading stats files is obviously better, as you discovered for yourself. I think I'm just about to understand your tool, and I wouldn't be surprised if that technology will become included in any upcoming all-in-one tools or even XviD codec versions (when the Sigma War is over one day)!

rui
26th August 2002, 08:49
Great work!

I think that one way of helping xvid is continuing with our efforts to help to develop ways of making even greater encodings with it.

milan
26th August 2002, 10:17
I've implemented idea from PerfectXviD into ffvfw and there is graph instead of list of numbers.
When I did this I tried to do something better: complete second pass simulation. It is very fast, because no compression is done and ffvfw just guesses second pass frame sizes based on first pass quantizer and frame size and requested second pass quantizer. This method is not very exact, so simulation results aren't exact too, but I think they can be helpful.
If you want to try this, you can download preliminary ffvfw binary from http://cutka.szm.sk/ffvfw/ and be sure you read the warnings about this alpha version.

rui
26th August 2002, 16:48
Well, i have been thinking (don’t worry, i didn’t saw any smoke coming out...:D) about the alt cc. parameters.

In the high parameter it says distance from the average frame size where the minimum relative quality will be applied
In the low parameter it says distance from the average frame size where the maximum relative quality will be applied

This are the parameters that i want to get, after configuring the alt cc. That is, after the 2nd pass encode, i want my avi to be created following the parameter i indicated the codec to steal bits (that’s the high setting-indicates the distance that the codec is allowed to go to get bits from those big frames, that’s why it’s called distance from the average where minimum relative quality will be applied), and the parameter that i indicated to the codec to atribute bits to small frames (that’s my low setting-indicates the distance that i want the codec to atribute bits to the small frames, so is the distance to the average where max quality will be applied).

BUT, i’m not sure if this line of thought is correct. If not, all this post is going to the trash can ;)

So, after analisyng the report that PerfectXvid 0,5 gave me:
PerfectXviD ver 0.5 by TheUnforgiven
unforgiven@soon.com

Total frames = 3570
eff. bits/(pixel*frame)= 0,797
3440 Delta-frame and 130 Key-frame

Key frames are excluded from the following stats :

Average frame size = 17.478 byte
Max frame size is +270% over the average = 64698 byte
Min frame size is -99% under the average = 101 byte

+275%
[ 2 frame 0,06%
+250%
[ 2 frame 0,06%
+225%
[ 14 frame 0,41%
+200%
[ 24 frame 0,70%
+175%
[ 53 frame 1,54%
+150%
[ 45 frame 1,31%
+125%
[ 59 frame 1,72%
+100%
[ 172 frame 5,00%
+75%
[ 140 frame 4,07%
+50%
[ 307 frame 8,92%
+25%
[ 559 frame 16,25%
--- Average frame size --- { 29 frame 0,84%
[ 898 frame 26,10%
-25%
[ 718 frame 20,87%
-50%
[ 209 frame 6,08%
-75%
[ 209 frame 6,08%
-100%

This are the results that i currently have in my source, not the ones that i want in my final avi. That’s why i will configure the alt cc, so i can change this, so the bitrate is distributed more accurately. Give bits where they are needed and take them from where it has loads of it.
One can see that the max frame size above average is 270% bigger, and the minimum frame size below average is around 99%.
If my previous interpretation mentioned in the begining of the post is correct, then i will config high at 270%, and low at somewhere near 90%-100%.
That’s were there are more doubts: the configuring of the low parameter.

As always all critics anybody has will be very welcome. I'm not sure if what i wrote is the truth. That's why we are all here, right? ;)

gldblade
26th August 2002, 20:54
>then i will config high at 270%, and low at somewhere near 90%-100%.

I think you've gotten it backwards. I think high should be 100 and low should be 270.

High pass sets a minimum frame size and low pass sets a maximum frame size. Following that and what you said here:

Max frame size is +270% over the average = 64698 byte
Min frame size is -99% under the average = 101 byte

Low pass is 270, high pass is 100.

rui
26th August 2002, 21:40
Well, i will try your settings and see what happens.

But i just tried Milan's new ffvfw, and the simulation mode works very well, at least for the small tests i have been doing. Only it has to be done outside of Vdub, or it will crash Vdub. But after experimenting with it, i take note of the values and then open Vdub and config the alt cc for 2nd pass.
In my The Replacements trailer test, it almost hit in it's predicted quantizers in respect to what i got in the end. :)

This could revolutionize the alt cc. choices!

EDIT: Hum.. My final average quantizer in one test was 7,2, but the simulation predited to be 7,95. A bit off, but nevertheless this new simulation mode is very promissing.

iago
26th August 2002, 21:59
@gldblade

Originally posted by gldblade

I think you've gotten it backwards. I think high should be 100 and low should be 270.
High pass sets a minimum frame size and low pass sets a maximum frame size. Following that and what you said here: Low pass is 270, high pass is 100. Actually I doubt that your interpretation reflects the situation here, and imho "you" have gotten it backwards. High/low values do not represent the min/max framesizes, but the distances (%) from average framesize where min/max relative quality will be applied as rui quoted. Maybe you've confused it with nandub high-pass/low-pass settings ;).

With the above report file results, I'd set the values as rui does: high:270/low:100

regards,
iago

OUTPinged_
26th August 2002, 23:27
@rui

Average frame size = 17.478 byte
Max frame size is +270% over the average = 64698 byte
Min frame size is -99% under the average = 101 byte



Try 125/50 for distances for that encode. Look if quantizers will be distroed better in that case.

Koepi
26th August 2002, 23:36
I still think high 500 / low 90 will give you a far more linear scaling experience - together with automatic min qual and strength at ~30 ;)

*hide*

Ah, and I think iago is right, don't mix up high/low distance with high- / low- pass like in nandub, it could give very strange results :)

Best regards,
Koepi

iago
27th August 2002, 00:28
To everybody,

I realized that the post I'd sent as reply to rui's interpretation of the high/low distance parameters was just mistaken, so it went to trash! ;) Those who have read it can forget it right away! ;)

best regards,
iago

gldblade
27th August 2002, 03:19
Oops. :p

Well, I guess it's a good thing I haven't tried this tool yet, otherwise I would have wasted quite a bit of time.

milan
27th August 2002, 07:48
Thank you for testing.

Originally posted by rui
Only it has to be done outside of Vdub, or it will crash Vdub.

It's fixed now.

OUTPinged_
27th August 2002, 08:28
glblade, there isn't anything wrong with tool.

it just gives you a framesize sheet.

"good settings" part is in planning stage :-)

FiW
27th August 2002, 11:13
:D
http://ukbdelo.ru/admin/altcc.gif

gldblade
27th August 2002, 14:59
glblade, there isn't anything wrong with tool.
it just gives you a framesize sheet.
"good settings" part is in planning stage

Well, I didn't say there was anything wrong with the tool. I meant I would have wasted time encoding with the wrong settings.

*EDIT*

Ah, I understand that graph now.

Yes, people, I'm stupid. :) I'll just quickly go back to obscurity now...

rui
27th August 2002, 15:03
IMHO the "tool" is great.
In my tests, the predicted quantizer distribution curve almost matches the final quantizer distribution curve, in spite that the average quantizer isn't exactly the same (in almost all cases i get a lower than expect average quantizer).
It allows one to experiment diferent settings and see the results almost instantly.

TheUnforgiven
27th August 2002, 19:15
@milan
great job
i sent u a PM, check plz

rui
27th August 2002, 21:52
Milan, I tried to experiment using your libavcodec, but always in the 2nd pass it crashes Vdub.
The only settings I changed were the quantization to MPEG, but it crashes also in H.263. All the rest was at default. It crashes both using b-frames and not using them.
But, like I said, only in the 2nd pass. If I try to encode using both quantizer and quality modes it works ok, even using b-frames.
I have WinXP Pro, AMD XP 1600+ OC to 1900+ (the problem isn’t from here), VIA KT133A chipset, Nvidia4 4200 Ti, all with the latest official drivers.
Decoding the resulting avi using your ffdshow seems fine.

Sorry to TheUnforgiven because this isn't related with his thread :o

milan
28th August 2002, 07:08
Second pass was broken. It should be fixed now, but it seems another problem is there... I will try to fix it today. Sorry, I think I should test it more before releasing it.

crOOk
28th August 2002, 19:19
I'd really like an option to cut off the credits. You could also add support for the debugview file format again. Thanx

TheUnforgiven
28th August 2002, 19:44
@crOOk
i'll do the credits soon. but y do u need the debugview format again? it is much slower.

Didée
29th August 2002, 16:31
Another suggestion, of which I think it could be quite useful:

A counter of how often the quants switched from 3 to 4 and from 4 to 3.

Koepi made a post lately, in which he mentioned that every switch from h.263 to mpeg quantizers - and vice versa - costs 140 byte extra!
(Man! I wasn´t aware of this - was it ever mentioned before?)

So, if we knew from 1st pass how often such switching happens, we would have some hint if, for the actual encoding, modulated quants are worth it, or if it would do more harm then good.

I´m no coder, so I can´t estimate how complicated (or easy) this calculation would be.
But the more info one can get, the better ...

crOOk
29th August 2002, 18:30
@TheUnforgiven
Because you could cut off the credits in the log file with a text editor.

TheUnforgiven
30th August 2002, 02:14
@crOOk
i'll do the credits when i come back from a short trip (2 days).
the older version is still available u can use it temporarly.
@didee
any thing with quantizers requires 2nd pass simulation. ask milan for that cuz he's doing that from inside the codec (using the codec stuff).

TheUnforgiven
2nd September 2002, 15:43
-just adds the possiblity to ignore the last x frames (credits).
get it here:
http://www.everwicked.com/forums/showthread.php?s=&postid=8495#post8495

Shayne
9th October 2002, 03:12
My understanding of alt curve was to rob from the rich to give to the poor. Take bits from high bit rate sence and give them to the low so that a more even quality is obtained.

Therefore to look at Purrfect xvid and assign high distance at a value above any frames makes little sense to me (please correct me if i am wrong since i am a little confused about the whole alt curve). Should you not at least set 10 percent of the frames to distribute to the low (some percentage anyway). If no frames can distribute down whats the curve compression really doing?

For me Koepi's defaults are fine and previous testing of alt curve indicates minnor quality improvements are possible at the pain of alot of testing and interupts in encoding.

PS this does have promise plz put a big <---- at the locations of low and high that should be used.

neo_born
12th October 2002, 06:51
Would someone like to explain how to use this tool and what to do with the info that it gives me ....step by step would be nice...though am not a total dunce!:D Did I mention ..pleeeeeeeease?

Koepi
12th October 2002, 10:47
neo_born,

save us the time and _READ_ through this thread.

You seem to have the attitude that education is for losers - but you're mistaken, I won't tolerate this lazyness as long as you request much work from us (to answer that "request" of yours would take up at least several minutes).

READ the threads you're posting in. _The whole_ thread.

Koepi