(XviD) Statsreader

[iago]

@Koepi,

BTW, can we use a 30082002-1 build 1stPass stats file (scaled down with statsreader 1.6) for 2ndPass with externalCC using the new 04092002-1 build? (sorry if it sounds pointless... but I guess there "are" some changes in the builds later than 30082002-1, so I feel a bit unsure and have to ask that question ;))

thanks for your patience :)
iago

[Koepi]

Nope, no changes in the core or first pass stuff were made since then :) You can safely use the old stats files...

Best regards,
Koepi

[iago]

@Koepi

Thanks for the immediate reply. So I "safely" started my second pass with 04092002-1 ;).

sevgiler (kindly greetings) ;)
iago

[Gannjunior]

@Koepi
Thanks 4 your answer.Statsreader 1.6 has solved my firend's problem: it scaled the statsfile correctly.
P.S. If you want,you can download the stats file zipped(1.9Mb)from here:
video.stats (http://www.geocities.com/maurizioferreira/first.zip )

ciao :D

[iago]

Yep, just for trying and fun ;):
(mission impossible :D)

Movie: Spartacus (3hr-08min) / PAL R2
576*240 / SimpleResize
target video size: 730000kb (1XCD with ogg audio)
Koepi's 04092002-1 build - statsreader 1.6

First Pass:
6.UltraHigh / h.263 / lumi masking / credits 20 quant.

Second Pass:
6.UltraHigh / h.263 / lumi masking / quantizers capped:2-6/2-16 / external curve compression with statsreader 1.6 / hi-lo: 0-0 with altCC disabled / payback proportionally / payback delay: 300 / credits 20 quant.

I'll post the results when finished; I'm looking forward to see how it will look! :D

kindest regards,
iago

[iago]

Hello everybody,

Here are the results of the above mentioned encode (using 04092002-1 build and external cc with statsreader 1.6):

(Aimed video size: 730000kb Final size: 730006kb !!! ;) Great...)

1stPass:
file size: 2191mb compressibility ratio: 3.104:1

2ndPass:

XviD Analyzer v0.12 by MoonWalker & MarcFD
e-mails : s_ilias@gmx.net & marc.fd@libertysurf.fr

Quantizers Analisis
---------------------

Quantizers Used For Movie :
------------------------------
Quant 3 Used : 7086 Times, Percentage Used : 2.54%
Quant 4 Used : 114744 Times, Percentage Used : 41.19%
Quant 5 Used : 111110 Times, Percentage Used : 39.89%
Quant 6 Used : 29966 Times, Percentage Used : 10.76%
Quant 7 Used : 3428 Times, Percentage Used : 1.23%
Quant 8 Used : 924 Times, Percentage Used : 0.33%
Quant 9 Used : 450 Times, Percentage Used : 0.16%
Quant 10 Used : 288 Times, Percentage Used : 0.10%
Quant 11 Used : 194 Times, Percentage Used : 0.07%
Quant 12 Used : 315 Times, Percentage Used : 0.11%
Quant 13 Used : 131 Times, Percentage Used : 0.05%
Quant 14 Used : 148 Times, Percentage Used : 0.05%
Quant 15 Used : 4100 Times, Percentage Used : 1.47%
Quant 16 Used : 5659 Times, Percentage Used : 2.03%

Average Quantizer Used for Movie : 5.082

Quantizers Used For Credits :
--------------------------------
Quant 20 Used : 4313 Times.

H.263 Quantization Type Used 282856 timed, Percentage Used : 100.00%

Quantizers prevented from rising too steeply 199 times


Intra-Frame (Key-Frame) Quantizers
------------------------------------

Movie
-------
Quant 6 Used : 1661 Times, Percentage Used : 98.05%

Credits
---------
Quant 20 Used : 33 Times, Percentage Used : 1.95%

Inter-Frame (P-Frame) Quantizers
------------------------------------

Movie
-------
Quant 3 Used : 7086 Times, Percentage Used : 2.52%
Quant 4 Used : 114744 Times, Percentage Used : 40.81%
Quant 5 Used : 111110 Times, Percentage Used : 39.52%
Quant 6 Used : 28305 Times, Percentage Used : 10.07%
Quant 7 Used : 3428 Times, Percentage Used : 1.22%
Quant 8 Used : 924 Times, Percentage Used : 0.33%
Quant 9 Used : 450 Times, Percentage Used : 0.16%
Quant 10 Used : 288 Times, Percentage Used : 0.10%
Quant 11 Used : 194 Times, Percentage Used : 0.07%
Quant 12 Used : 315 Times, Percentage Used : 0.11%
Quant 13 Used : 131 Times, Percentage Used : 0.05%
Quant 14 Used : 148 Times, Percentage Used : 0.05%
Quant 15 Used : 4100 Times, Percentage Used : 1.46%
Quant 16 Used : 5659 Times, Percentage Used : 2.01%

Credits
---------
Quant 20 Used : 4280 Times, Percentage Used : 1.52%



Frame Analisis
----------------

Number Of Intra-Frames (Key-Frames) : 1694
Number Of Inter-Frames (P-Frames) : 281162
Total Number Of Frames : 282855

0.60% of the Movie is Intra-Frames (Key-Frames)
99.40% of the Movie is Inter-Frames (P-Frames)

Size Analysis
----------------

1-Pass Size : 2297538270 Bytes or 2243689 KBytes or 2191 MBytes
Scaled Size : 740575358 Bytes or 723218 KBytes or 706 MBytes
Actual Size : 740574642 Bytes or 723217 KBytes or 706 MBytes

Usefull Statistics
------------------
Compressibility : -37.08%
Relative Quality of XviD avi : 39.35%
Absolute Quality of XviD avi : 90.75%


And, interestingly, the movie is "not" unwatchable; on the contrary it "is" a decent encode, perfectly watchable, in spite of the average quantizer ~5, with a pretty and smooth playback using Nic's decoder filter at strength 4 (full deblock) :). (15 and 16 quants in the report are only used at the beginning (first 4 or 5 minutes) of the film where long durations of only black screen is present with some still scenes occuring from time to time.

EDIT: Well, of course it is far from being "perfect and flawless" ;), but it is still "perfectly watchable" without disturbing you much; and even more "watchable" if you add some noise using ffdshow ;). In short, it is absolutely "not" a "messed up and unwatchable" encode, which really surprised me.

best regards from iago ;)
(who is much happier now with linear scaling)

[Koepi]

Hehe, interesting that this doesn't mess up the movie :)

Btw., internal scaling with regular CC at 0/0 or altCC.... you know the settings... - should result in the _same_ result. Now it's interesting if there still is a difference.

I need another movie to encode, can't see the matrix for some hours ;)

Regards,
koepi

[iago]

Originally posted by Koepi
Btw., internal scaling with regular CC at 0/0 or altCC.... you know the settings... - should result in the _same_ result. Now it's interesting if there still is a difference.@Koepi,

I encoded that movie only once, with regular cc at 0/0 with the above settings. I didn't use "altCC with strength 0 or mrq 100 and the other parameters default", which as you pointed out should give the same result as regular cc at 0/0 ;). So, there is nothing confusing or uncertain I guess? ;)

(And the encode mentioned above was a fresh two pass with 04092002-1, and statsreader 1.6 for scaling down the curve.)

kindest regards,
iago

[Koepi]

oh, the difference i meant was:

there should be no difference if you use internal or external scaling.

I'll encode something in some time to compare this.

Best regards,
Koepi

[iago]

Originally posted by Koepi
oh, the difference i meant was:

there should be no difference if you use internal or external scaling.

I'll encode something in some time to compare this.@Koepi

Oh, I see now ;). Sorry for the misunderstanding ;).

ciao,
iago

[Emp3r0r]

Originally posted by iago
quantizers capped:2-6/2-16I have yet to understand the purpose behind capping quants. Why not let xvid do it's magic?

[Koepi]

Originally posted by Emp3r0r
I have yet to understand the purpose behind capping quants. Why not let xvid do it's magic?

Well, sometimes magic fails and therefore some safety-belts ;)

Regards,
Koepi

[iago]

@Emp3r0r

Maybe a sort of caution or just a habit, I don't know ;). In fact, I usually prefer 2-4/2-8 for my rips, but this one was a real hard case (3hr-8min). Also, as I remember, somewhere Koepi had suggested 2-6/2-16 for 1CD rips and 2-5/2-16 for 2CD, if I'm not mistaken.

regards,
iago

EDIT: Koepi, I cannot believe it anymore !!! ;). Is this really a coincidence?! These simultaneous posts! ;)

[MaTTeR]

Originally posted by Koepi


Well, sometimes magic fails and therefore some safety-belts ;)

Regards,
Koepi
Hehehe...I totally agree. I remember Koepi stating that sometimes he could see the higher quant frames, well so can I and I suppose were not the only one's. I always cap the quants just as Iago mentioned(2-4/2-8). Admittingly the majority of my rips are targeted for either 970MB(XCD/99min) or 1410MB on 2CD's with full 5.1 AC3. For single CD rips I might fall back to something a little more conservative such as 2-6/2-12. Of course capping the quants also requires me to encode the movie several times for best results while I tweak but that's not really an issue with dual XP 1600's at less than 55mins per pass with TemporalSoften around 640 resolution. Perhaps I wouldn't be so eager to tweak the quants if I was still running dual PIII 800s:D

[MoonWalker]

@Matter

Out of topic a little. For 2-cd rips you use MPEG quant? Cause I use too but I get many MPEG arifacts.Can you post your avs ?? If you don't mind you can PM me, so we don't post on the topic..

Thanks,
MoonWalker

[ookzDVD]

@Koepi,

I've been trying v1.5 and the good news is the "Exit error" issue
is gone. Thank you.

[Maurizio]

hello everybody.
It's a lot of time I follow this forum and this is my first post.
I'm the friend of Gannjunior, who had the problem with negative results.
just a little question:

Originally posted by Koepi
oh, the difference i meant was:
there should be no difference if you use internal or external scaling.

Now I don't understand anything.
do you mean :
1) if I use external scaling, I get exactly the same results either I use
regular cc with 0/0 or alt cc whith known values
2) if I use regular cc with 0/0 or alt cc whith known values,
I get the same results either using internal or external scaling.
If this is the case, what's the point of using external scaling at all (since this requires one of the above cc settings) ?

Regards
Maurizio

[Koepi]

internal and external scaling should give the same results when using regCC with 0/0.

I started to add the scaling ability to the StatsReader as it was fun to code, and as a test.

Now we have the doubt that it _is_ the same as it should be. I'm conducting a test today (if the quantizer distribution differs between internal and external scaling, there's something wrong with the code :( )

You're right, if there's no difference, no need to use the statsreader for scaling. But if there is....

Well, let's ope everything is fine. My first small test showed a difference, but the statsfile might have been broken as it was from a crashed first pass session.

Regards,
Koepi

[Maurizio]

So if I'm not wrong, you are just experimenting with linear scaling
before passing to the more difficult task of unlinear one ?
btw, broadly speaking , should unlinear produce better results (whatever this might means) than not linear ?
regards
Maurizio

[TripleA]

I did some encoding tests of "Soul Survivors". Using Koepi's Sept. 4th build. All with a target size of 728984KB. All reasonably close (under 1MB off). Here are the analyses (XDA v0.11. v0.12 doesn't work for me...):

First pass h.263, 2nd pass modulated HQ, internal CC:

Quantizers Analisis
---------------------

Quantizers Used For Movie :
------------------------------
Quant 2 Used : 63169 Times, Percentage Used : 54.85%
Quant 3 Used : 50667 Times, Percentage Used : 44.00%
Quant 4 Used : 1298 Times, Percentage Used : 1.13%
Quant 5 Used : 30 Times, Percentage Used : 0.03%

Average Quantizer Used for Movie : 2.463

Quantizers Used For Credits :
--------------------------------
Quant 20 Used : 7978 Times.

MPEG Quantization Type Used 9306 timed, Percentage Used : 7.56%

H.263 Quantization Type Used 113836 timed, Percentage Used : 92.44%

Quantizers prevented from rising too steeply 100 times


Intra-Frame (Key-Frame) Quantizers
------------------------------------

Movie
-------
Quant 2 Used : 1447 Times, Percentage Used : 94.51%
Quant 3 Used : 24 Times, Percentage Used : 1.57%

Credits
---------
Quant 20 Used : 60 Times, Percentage Used : 3.92%

Number Of Consecutive I-Frames : 0

Inter-Frame (P-Frame) Quantizers
------------------------------------

Movie
-------
Quant 2 Used : 61722 Times, Percentage Used : 50.75%
Quant 3 Used : 50643 Times, Percentage Used : 41.64%
Quant 4 Used : 1298 Times, Percentage Used : 1.07%
Quant 5 Used : 30 Times, Percentage Used : 0.02%

Credits
---------
Quant 20 Used : 7918 Times, Percentage Used : 6.51%



Frame Analisis
----------------

Number Of Intra-Frames (Key-Frames) : 1531
Number Of Inter-Frames (P-Frames) : 121611
Total Number Of Frames : 123141

1.24% of the Movie is Intra-Frames (Key-Frames)
98.76% of the Movie is Inter-Frames (P-Frames)



Size Analysis
----------------

1-Pass Size : 932481456 Bytes or 910626 KBytes
Scaled Size : 743497266 Bytes or 726071 KBytes
Actual Size : 743741918 Bytes or 726310 KBytes

Usefull Statistics
------------------
Compressibility : 79.76%
Relative Quality of XviD avi : 81.19%
Absolute Quality of XviD avi : 98.61%

First pass h.263, 2nd pass modulated HQ, external CC (StatsReader v1.5):

Quantizers Analisis
---------------------

Quantizers Used For Movie :
------------------------------
Quant 2 Used : 46383 Times, Percentage Used : 40.28%
Quant 3 Used : 68661 Times, Percentage Used : 59.62%
Quant 4 Used : 111 Times, Percentage Used : 0.10%
Quant 5 Used : 9 Times, Percentage Used : 0.01%

Average Quantizer Used for Movie : 2.598

Quantizers Used For Credits :
--------------------------------
Quant 20 Used : 7978 Times.

MPEG Quantization Type Used 8098 timed, Percentage Used : 6.58%

H.263 Quantization Type Used 115044 timed, Percentage Used : 93.42%

Quantizers prevented from rising too steeply 0 times


Intra-Frame (Key-Frame) Quantizers
------------------------------------

Movie
-------
Quant 2 Used : 1453 Times, Percentage Used : 94.53%
Quant 3 Used : 24 Times, Percentage Used : 1.56%

Credits
---------
Quant 20 Used : 60 Times, Percentage Used : 3.90%

Number Of Consecutive I-Frames : 0

Inter-Frame (P-Frame) Quantizers
------------------------------------

Movie
-------
Quant 2 Used : 44930 Times, Percentage Used : 36.95%
Quant 3 Used : 68637 Times, Percentage Used : 56.44%
Quant 4 Used : 111 Times, Percentage Used : 0.09%
Quant 5 Used : 9 Times, Percentage Used : 0.01%

Credits
---------
Quant 20 Used : 7918 Times, Percentage Used : 6.51%



Frame Analisis
----------------

Number Of Intra-Frames (Key-Frames) : 1537
Number Of Inter-Frames (P-Frames) : 121605
Total Number Of Frames : 123141

1.25% of the Movie is Intra-Frames (Key-Frames)
98.75% of the Movie is Inter-Frames (P-Frames)



Size Analysis
----------------

1-Pass Size : 932481456 Bytes or 910626 KBytes
Scaled Size : 743466503 Bytes or 726041 KBytes
Actual Size : 743715324 Bytes or 726284 KBytes

Usefull Statistics
------------------
Compressibility : 79.76%
Relative Quality of XviD avi : 76.97%
Absolute Quality of XviD avi : 98.20%

First pass MPEG, 2nd pass modulated HQ, external CC (StatsReader v1.5):

Quantizers Analisis
---------------------

Quantizers Used For Movie :
------------------------------
Quant 2 Used : 61594 Times, Percentage Used : 53.48%
Quant 3 Used : 52631 Times, Percentage Used : 45.70%
Quant 4 Used : 937 Times, Percentage Used : 0.81%
Quant 5 Used : 2 Times, Percentage Used : 0.00%

Average Quantizer Used for Movie : 2.473

Quantizers Used For Credits :
--------------------------------
Quant 20 Used : 7978 Times.

MPEG Quantization Type Used 8917 timed, Percentage Used : 7.24%

H.263 Quantization Type Used 114225 timed, Percentage Used : 92.76%

Quantizers prevented from rising too steeply 11 times


Intra-Frame (Key-Frame) Quantizers
------------------------------------

Movie
-------
Quant 2 Used : 1397 Times, Percentage Used : 91.13%
Quant 3 Used : 56 Times, Percentage Used : 3.65%
Quant 4 Used : 20 Times, Percentage Used : 1.30%

Credits
---------
Quant 20 Used : 60 Times, Percentage Used : 3.91%

Number Of Consecutive I-Frames : 0

Inter-Frame (P-Frame) Quantizers
------------------------------------

Movie
-------
Quant 2 Used : 60197 Times, Percentage Used : 49.50%
Quant 3 Used : 52575 Times, Percentage Used : 43.23%
Quant 4 Used : 917 Times, Percentage Used : 0.75%
Quant 5 Used : 2 Times, Percentage Used : 0.00%

Credits
---------
Quant 20 Used : 7918 Times, Percentage Used : 6.51%



Frame Analisis
----------------

Number Of Intra-Frames (Key-Frames) : 1533
Number Of Inter-Frames (P-Frames) : 121609
Total Number Of Frames : 123141

1.24% of the Movie is Intra-Frames (Key-Frames)
98.76% of the Movie is Inter-Frames (P-Frames)



Size Analysis
----------------

1-Pass Size : 1063509869 Bytes or 1038583 KBytes
Scaled Size : 743466744 Bytes or 726041 KBytes
Actual Size : 743398910 Bytes or 725975 KBytes

Usefull Statistics
------------------
Compressibility : 69.90%
Relative Quality of XviD avi : 80.86%
Absolute Quality of XviD avi : 98.58%

[iago]

Originally posted by Koepi
internal and external scaling should give the same results when using regCC with 0/0.
Now we have the doubt that it _is_ the same as it should be. I'm conducting a test today (if the quantizer distribution differs between internal and external scaling, there's something wrong with the code :( )@Koepi,

Any (good) news about your test ;)?

best regards,
iago

[Koepi]

Just finished the test:
Comparison of linear external vs internal curve compression:

Target size 599.000kb.

mpeg2dec.dll, mpeg2dec-PP6, mpeg2dec-PP6,
h263/h263 h263/h263 h263/h263
ext.CC ext.CC int.CC lin.

Q:2:1324 Q:2:1200 Q:2:1210
Q:3:95354 Q:3:103581 Q:3:103755
Q:4:84936 Q:4:78063 Q:4:77890
Q:5:4006 Q:5:2991 Q:5:2991
Q:6:415 Q:6:195 Q:6:188
Q:7:69 Q:7:70 Q:7:88
Q:8:25 Q:8:38 Q:8:35
Q:9:17 Q:9:36 Q:9:22
Q:10:34 Q:10:14 Q:10:9
Q:11:8

599.012kb 599.024kb 599.114kb


So it isn't behaving the same. There is a small difference which could be causing this: Foxer is redistributing the rounding error, which the statsreader doesn't (thus you never hit you target rate to the byte with statsreader curve scaling [I mean the report at the end of the scaling]). But in the end it seems to hit the desired rate more accurately this way, strange.
I'll ask foxer when he's online again if he has some other ideas. Good thing is that the results aren't dramatically different.

Now I'm going to the visual check.

Regards,
Koepi

[Dali Lama]

Hello!

I have tried to perform some of the tests suggested by you guys. I am having difficulty with the StatsReader 1.5. When I open my first pass stats and enter a reasonable file size I get a a msg saying scaled down to -919.06 MB?? Also, the compression ratio is negative... -2.673:1???

I have used an earlier beta before, but a negative result did not occur.

Thanks to all and good day,

Dali

[Koepi]

Dali,

fetch yourself statsreader v1.6, some posts before this one.

It has the issue with first pass size >2GB solved, which resulted in the behaviour you observed.

Thanks,

best regards,
Koepi

EDIT: the post is on page 8, the link to the attachment is http://forum.doom9.org/attachment.php?s=&postid=175804

[Dali Lama]

Thank You Koepi,

Issue resolved...testing resumed :)

Dali

[Sharro]

Cmon Koepi...my encode came out 6KB SHORT.....tsssccc tsccc....

Just kidding... and taking the opportunity to say Thx!! :-)))

Take Care,


Sharro

[iago]

@Sharro

LOL! Sense of humour is something I really appreciate ;). Yes, XviD is absolutely outstanding, indisputable winner, in predictability. (Imho, this point should also be included in all sorts of codec comparisons or discussions (just like Doom9 does), besides its rivaling or surpassing quality.

regards,
iago

[Koepi]

StatsReader 1.7 now tries to not scale "minimum sized" frames (e.g. 96bytes or 101 bytes frames, depending on the resolution).

For those frames a compensation is built-in now, this should avoid problems where longer low bitrate scenes could stack up overflow and thus result in some highly quantized frames which are simply spoken ugly.

I hope you like it, please test it! (I added info about those conditions to the output, too... I think i could make a "full report" version which does a quantizer distribution simulation, too ;) ).

Find the new version attached,

thanks,

and best regards,
Koepi

[iago]

Wowww, just great man! ;)
Koepi's never stopping, never giving up! ;)

thanks for the new toy,
ciao,
iago

[Koepi]

"What's the worst that could happen?", 640x352...

targeting: 631.432kb, reached: 631.456kb.
First pass size: 1.210mb

[1016] Quantizer distribution for 2nd pass:
[1016] Q:2:22
[1016] Q:3:46194
[1016] Q:4:87069
[1016] Q:5:1683
[1016] Q:6:4
[1016] Q:7:3
[1016] Q:8:3
[1016] Q:9:2
[1016] Q:10:2

I think it's amazing that "only 14 bad frames" are left .)

I didn't test that movie with other settings, but I might do on the "timemachine" which is the next movie I'm trying to encode ;)

Just wanted to share my result...

Best regards,
Koepi

P.S.: maybe i must finetune the formular to get the min. frame size in respect to macroblocks... it's somewhat weird, it's still no "linear" function if i subtract the avi frame overhead, and I don't see what I'm missing :-/

[zulu]

I didn't test that movie with other settings, but I might do on the "timemachine" which is the next movie I'm trying to encode


great! :)

i'm wondering how this movie will turn out for you.
i tested it both with internal and extnernal CC but can't get it to look good.

There were too many macroblocks for my taste.
Please have in particular a look at the scene at frame pos. ~ 114700.

Looking forward for your results :)

[rui]

One question:

When using stastreader, one shouldn't use the alt.cc or normal cc values. That's why it's adviced to input, for example in default cc, high/low to 0, so one gets a linear curve.

BUT, i did some tests days ago using the statsreader AND alt cc, and got diferent results than using only statsreader, or only alt cc.
In this cases the codec is using what type of curve to distribute the bits? Is the alt cc getting priority over statsreader stats file, or the other way around? Or a mix of both (since i am getting diferent results when using both, like i said)

P.S. I am having problems in my comp at work, so i can't test here anymore :(. I can't reproduce those tests i mentioned above.

[Koepi]

@zulu:

my first pass size is 1.365mb, see the scaling results attached. It may be that you ran into trouble with minsize frames before that scene...

@rui:
it takes the downscaled values from the stats2-file. It then applied curve correction with the overflow and the desired rate given by that statsfile and thus results into something different.

Regards,
Koepi

[iago]

Originally posted by Koepi
StatsReader 1.7 now tries to not scale "minimum sized" frames (e.g. 96bytes or 101 bytes frames, depending on the resolution).

For those frames a compensation is built-in now, this should avoid problems where longer low bitrate scenes could stack up overflow and thus result in some highly quantized frames which are simply spoken ugly.That's simply great. One step closer to the solution and to excellence ;).

Originally posted by Koepi
...I think i could make a "full report" version which does a quantizer distribution simulation, too ;)In my opinion, that would really be very useful, another great option for the outstanding statsreader ;).


Thanks again for your never-ending efforts and great works,
best regards,

iago

[Koepi]

I'm testing with "from hell" now, a very compressable movie. Good that I found the lanczos3.dll resize filter on the avisynth forum, this made it possible that the movie has to be compressed in the second pass at all (640x272 neutral bicubic came out at just 601000kb, now I've to downscale from 631000kb to 621000kb ;) ).

This movie has long "scenes" with min framesize, and the statsreader curve works like expected, no stacked up overflow, no degradation due to that!

Hm. On the further development, thinking of this movie the simulated second pass will be very inaccurate as it would show that all frames get quant. 2 which isn't true. But it gives an idea at last...

Ok, I'll think about writing a report-file, this should be very useful for further checking out what could be done.

Best regards,
Koepi

[toliman]

i also have 'from hell', at a slightly lower res, sitting on the HDD waiting to be encoded. I replaced the older codec i had with the 04/09 build, with the new mod.HQ experimental settings, which missed the indicated size by about 40mb :scared: . Looked crisp though, which was a nice surprise, statsreader told me i was getting very close to the first pass, which is another surprise. a really good compressible movie that fits on 1cd. using the statsreader didnt gain any more accuracy.

(i might need different first-pass stats for each kind of quantizer, which i didnt think was necessary, the thought just came to mind then. is this a correct assumption?)

i've tried to follow the (wandering) discussion in this post, but there is not much info on mod.HQ, which i find more intrguing than modulated/h.263/mpeg as something purely experimental or as a recommended setting (what info i did read on mod.HQ, got lost about 40 posts ago)... anyway, i'll see if i can do first and second pass in mod.HQ instead of first pass with h.263. (i had the impression mod.HQ was trying the 'nandub approach' to low-motion-hi-motion quantisers that made SBC awesome over regular low-motion encoding.) i'm willing to try anything from b-frames to psy/lumi enhancements to gain a small amount of visual quality, ive got the HDD space and the DVD's to play with to try it out.

The other problem with 'from hell' is the credits are blood-red, what does setting "encode credits in greyscale" do, modifying YUV intensity/values, i.e. what effect does non-white/black sequences make to greyscale credit encoding ?

also, while on the subject of encoding credits in first/second pass, is it better to use something a bit easier on the eyes than q.20, like q.12, and duplicate the same settings in 2nd pass? does this effect statsreader's accuracy or is this no problem.

(you can also just point me towards a FAQ on these questions, if there is one)

[Koepi]

Unfortunately all those infos you're searching for a spread ovr diferent threads so I like to urge you to use the search function, view the results in a new tab/window and learn about those things.

I don't understand why mod. HQ would destroy size predictability - it were other settings you were using. Use "load defaults" and you're mostly fine :)

if you use mod quant or mod. HQ, you will gain nothing from using it in the first pass. But read about it in the thread where we tested that. (Hint: first pass is done at constant quantizer 2....).

Uff. You didn't read the XOE or the help file. If you use fixed. quant credits, you have to use the same settings for both passes, else your final filesize won't hit the targeted size. (I assume that's the error you did above and got 40mb oversized...). So, read the XOE even if it's outdated a little.

Ok, I lost motivation in going on and explain things that are explained all around this place once more, condensed in one post, so I'll simply stop the reply now ;) I hope you can understand that it's very frustrating typing all the stuff again and again (and please read the forum rules - No. 1 is VERY important :) ).

Regards,
Koepi

[toliman]

Originally posted by Koepi
I don't understand why mod. HQ would destroy size predictability - it were other settings you were using. Use "load defaults" and you're mostly fine :)


im redoing a lot of encoded files recently, so i'm reworking old assumptions and asking questions, the current encodes dont seem to be working so well with the defaults i had. there is a 40gb backload of dvd's & divx re-encodes to go through, so im a little uhh ... curious on the whole process with mod.HQ and the new external CC.

e.g. a 3cd version of the Thin Red Line (avg 1100kbps) turned out with massive segments between keyframes, choosing modulated quant in the 2nd pass, 120 frames taking up 7-9mb or more... was hard to cut around the massive 200 frame, 12mb sections between I-frames to fit neatly. Then, i take the opposite extreme and encode a 80 min movie, and it turns out to be 20mb undersized modulated quant. 2nd pass.

i'm taking the weekend to go through and find any method to actually hit the target size i enter in, it seems like an odd goal for 9-10 hours of work to aim for +/- 500kb, instead of +/- 10-20mb from the intended size.

[Koepi]

Sorry pal,

your settings are definatly messed.

_do_ as I wrote and USE the search (I'll attach a picture how to do that later). And, with all respect *caugh*, this is OFF TOPIC to the MAX.

Btw. - I hit in the bounds of +/- 24kb max. variation from my target size. So rethink your settings properly after reading all available documentation.

Did you read forum rule 1?

Koepi

[Easy123]

@Koepi

I also did a Test with "From Hell" and the movie looked damn good until it came to the end where "Jack the Ripper" puts the Heart into the Fire... The whole scene blocked up with ugly Macroblocks... I used MPEG Quantizers in first pass, and MOD HQ in Second Pass Ext (Quant 2-6 & 2-16) (stats File done with Statsreader). Do you know an Answer why it suddenly blocked up??? never had this prob before. All movies came out Perfect....


Cu
Easy123

[manono]

Hi-

Perhaps you started the lower quality End Credits too early?

And, of course, fire is hard to compress. Did you run DeBugView while encoding so you could check what quants were being assigned to the fire scene?

[Easy123]

No haven´t run debugview but guess I´ll do that and take a look....


Easy123

[Koepi]

The movie looks perfectly here no matter what I do.

Easy, credits range is from 166376-176028 ;) Please rechceck that, manono is probably right.

Best regards,
Koepi

[Easy123]

Well, now it works... Guess I must have typed a wrong number into statsreader, without seeing it... (am i dumb, doh :D )

[Koepi]

Results for Zoolander, lanczos3 resize to 640x272, first pass size 1233mb, target size 642999kb, reached 643020kb.

[1016] Quantizer distribution for 2nd pass:
[1016] Q:2:125
[1016] Q:3:46806
[1016] Q:4:70631
[1016] Q:5:2089
[1016] Q:6:9

No quantizers >6, statsreader now seems to work better than internal CC! :)

Just wanted to share that again :)

regards,
Koepi

[iago]

FULL EDIT: Sorry. Totally cancelled! Using Statsreader 1.7 and ColorYUY2 together (for a visual test, regarding the treatment of blockiness in black areas as well) is completely contradictory. Statsreader 1.7 itself is trying to do this, not scaling the minimum sized frames! Seems I got a bit confused recently :(.

[MoonWalker]

@Koepi

Hi,
Just an idea. Why don't you put the scaling code of the stats reader into XviD's scaling code? For example, at normal CC if we use high/low : -1/-1 could triger your scaling code..

Just a thought,

Excellent work

MoonWalker

[Koepi]

@MoonWalker:

That's exactly what I plan to do, but there is still the problem with the propper min. frame size formula... if that's solved, I'm going to hack that into 2pass.c (i thought I mentioned that some times now ;) ).

Else it would be "useless" work (well, not really, you can always use StatsReader ;) ).
We're investigating on this, but it seems that only -h and Eduard seem to care about that issue... let's see what happens, I'm definatly way out of "simple maths" training and don't find the magic formula.

Best regards,
Koepi

[manono]

Hi-

TheWEF's formula for Minimum Bit Rate in his Modified Stats File for DivX3.11 was 25% of the Average Bit Rate, with a bottom at 270. So, once the Bit Rate got above 1080, the minimum started to rise. It always seemed reasonable to me. Plus, he gave us the option to adjust it ourselves (useful for dark movies done for 1 CD).

[Koepi]

Thanks for the hint, but that's not what I've been looking for. XviD is _not_ nandub/divx3.

I now have the formula for pframes min. size, for iframes the error gets distributed much "finer" so a special treatment here isn't necessary IMHO (and very complicated - not possible without access to the bitstream itself... the _final_ bitstream with a possibility to reencode the frame when necessary...).

I'll implement it into the StatsReader soon and will call it v1.8.

Thanks for the help,

and best regards,
Koepi