Cvs 2002-02-26

[-h]

Added Foxer's asymmetric curve compression code. You can go back to the old "symmetric" code by setting the high and low variables to the same value (i.e. 25%). The first line in dbgview's output is the "average asymmetric frame size", above which the "high" variable takes effect, below which the "low" variable takes effect.

Also fixed an overflow bug.

Still haven't enabled/disabled any new controls though :)

-h

[kastro68]

where can i go to download build 26/2/2002?

[-h]

No idea, whenever the very nice binary site ops upload it :)

I just post stuff to CVS. Yes I am lazy.

-h

[sierrafoxtrot]

*koff koff* ... er ... nic?

:D

cheers,

SF

[rui]

*koff koff* and *KOFF* ... er ... Nic?

Or, Koepi? *koff* modulated quantization *koff*?

:D :D

[Nic]

Where d'ya think? :D
(I uploaded your first bug fix under the name 26_02_02 so this one is 26_02_02 -part 2)

www.freewebz.com/xvid
or
xvid.stormpages.com

-Nic

[kastro68]

Only did one test clip...error message in vdub job control. Actually i tried it on 2 different vids.

I used P lock 2-6, I lock 1-6

output vid size 5000

H.263

FourCC Xvid

[-h]

Only did one test clip...error message in vdub job control.

This is a virtualdub issue - it stores the codec state data that XviD returns, but it appears unable to deal with the (ever-growing ;)) amount of data we're storing. Well, that happens when you add that many controls to a codec.

Well I don't want to speak too fast - does the new build encode fine when not using job control? Or is job control all that crashes it?

-h

[kastro68]

Pretty sure it is a bug now, i just tried encoding a divx4.12 to xvid...I CAN'T EVEN DO THE FIRST PASS. Same quant settings as before.

[kastro68]

Crashes even though i wasn't using job control

[kastro68]

Video Compression error: unspecified error (-100)

it doesn't work with quantizers of 1-31 for both I and P either

[-h]

Very odd.

I'll upload the dll I've built over here.. hang on.

-h

[Koepi]

Well, since -h did incorporate some changes I had to wait until 12:40h to checkout the CVS.

Follow the XviD link in my signature to find my newest binary with included source modifications...

Quant_type modulation is back in :) Now combined with the whole curve-treatment stuff it's definatly the best codec available! :) (I don't need to use nandub anymore I'd say!)

Best regards,
Koepi

[kastro68]

Well i know Rui would definitely be happy :)

[-h]

@kastro68 - I've uploaded 2 dlls to http://briefcase.yahoo.com/hstink - xvid.zip is just a standard compile, xviddebug.zip is a dll with extra debug output.

It sounds like something is stuck open or locked on your pc - I'd advise rebooting, then trying Nic's dll again. If that doesn't work, try my xvid.zip. If that doesn't work either, try xviddebug.zip and upload the dbgview log so we can see exactly what's falling over.

-h

[kastro68]

I am starting to think that the problem is coming from my side because Koepi's build had the same problem.

[kastro68]

Well I haven't shut down my comp for over a week, that could be the problem.

Rebooting

[kastro68]

Just tried Koepi's build after teh reboot, same problem

[kastro68]

Just tried kilted yaksman's build, same problem

trying debug version now

[kastro68]

I am sure the problem is from my side...I went back to Nic's 21/2/2002 build and it reports the same problem.

Hope i don't have to reformat windows

[kastro68]

Ok, i'm gonna try it on my other comp to see if the same problem occurs. ATM Nic's 21/2/2002 build works fine on it...Just reluctant though because it may render that comp unable to encode to Xvid too.

[kastro68]

Both Koepi's 26/2/2002 and Nic's 26/2/2002 build worked fine on the other comp.

I think i better try divX4.12 on the Amd to see if that still works.


Would anyone know what could be wrong?

[Koepi]

Since I use a Duron700 here it's definatly no AMD error...

Regards,
Koepi

[rui]

Originally posted by kastro68
Well i know Rui would definitely be happy :)

You can bet it!!:D

Koepi, if you were a girl, i would give you a BIG virtual kiss!! :D :D

Just making a first pass using Koepi's build. For now a didn't got any errors.

When this is done, i will tell you guys about the results.

[kastro68]

Works on AMD now!!!!

Sorry about that.

It seems that i saved the 1st pass stats to "h:\smj\test" which didn't work!

This could be a bug. But only minor

[kastro68]

I feel really bad now :(

Sorry for alerting everyone...I'm just clumsy sometimes

[rui]

Kastro68, don't give it another thought.

I for one am always glad to read your takes on this matters.
We are all uncovering new ground here (that is, apart from Nic, -h and Koepi, of course ;)

[Koepi]

Ok, ripping the matrix now to make sure that 1CD still works and looks really nice (well, if quant 2,3,4 get used most often i am SURE it still looks good.).

SCNR

Koepi

[Nic]

Kastro: Dont worry about it, I should have read the thread (Doh!) I too have got the general error (-100) for setting the stats folder to one that is invalid or doesn't exist :)

Tell us how it goes Koepi! :)

-Nic

[Koepi]

Ok, if you're so keen on my progress, I won't deny to tell ya... ;)

3 minutes still left until the vid+audio are on HD.
After that DVD2AVI and setting up my .avs will take another 15 minutes at least... ;)

Regards,
Koepi

[Nic]

Well I didn't quite mean a blow by blow account :D ;) LoL

Take Care,
-Nic

[Franko30]

Originally posted by Koepi
(well, if quant 2,3,4 get used most often i am SURE it still looks good.).

Koepi

Hi again,

Now it's time for my first really dumb question.

As my searches for "Foxer" and "curve compression" only delivered results for what I already have read and taking into account my "only Quant 5-6" results with curve compression standard settings described in my post in the "CVS 2002-02-25" thread I ask myself (and all of you who seem to get great results):

How do the curve compression settings really work and how are the settings to be used to get good (Quant 2-4) results for the low-motion scenes?

Sorry I have to ask such a dumb question but from what was said in this forum so far concerning the curve compression, I just didn't understand it.

Frank

[rui]

Well, i have endend my test using Koepi’s latest build, again using a trailer.
But before i post my results, i want to say something:

I have been reading some posts saying that testing the new builds using trailers isn’t a very good way of doing things, that we should only post results of tests made using full movies.
I have to say that i AGREE with this. Of course a full movie test is MUCH better than a trailer test.
The reason i post so much tests using trailers is: most of the time i am at work, where i have a lousy computer (P2 350 with no DVD-ROM with 64 MB). The only way i found out of conducting tests here is by ripping a trailer in home, then save it ot a CD-RW, the copying that trailer to my Pentium 2, and using it to make the tests.
For obvius reasons i can’t bring from home a full movie (and it would take forever to encode a full movie with a Pentium 2). So i play with what i have ;)

Ok, now for the results:

First pass using H.263, 5 motion search precision, 1-31. For second pass i used Foxer’s default setups, modulated quantization, 1-31, I – frame lock 2-5 with smooth quantizer fluctuaction enabled.

I found out that both Koepi’s build and Nic’s build are very good. With this small test i can’t see much diference between them.
But... it could be just me, but i believe that in close ups, Koepi’s build seems a little more detailed, more sharper. If this is correct, probably is because of the modulated quantization, that i like so much. :)

Ok., there you have it, either with Nic’s or Koepi’s, we are in very good hands.


P.S. After reading Franko30 post, i too would like to know how can one improve is encodings with the new foxer’s settings . Is there any site that has those settings explained? I already read a post from –h explaining them, but i would like to know the experts opinions on some settings that would be better for, say, 1 cd encoding and 2 cd encoding.
I now that with diferent movies the settings could change a lot, but maybe you guys could give us some values to start with, like you did with the I-frame lock (2-5 for 1 cd and 2-3 for 2 cd’s, if i remember correctly).

[sierrafoxtrot]

@rui

hey there, read in a nandub forum somewhere that for asymmetrical curve compression, low values of 15% and high of 25% give decent results. whatever that means ... i'll have to wait to get home before i can test anything out (itching to encode) :D

[Koepi]

@ all:

for my progress.... doing the first pass now, looks promising (no noise filter, want a "speed up test", there's a little noise on the movie :/ )

for curve compression questions:

Read nandub options explained and doom9's guide about nandub. There you can find an explanation of curve compression :)

(just as short summary:
the stats file is more or less a bitrate distribution curve. in second pass, we just did scale that down to achieve our desired filesize directly, on a mostly per-frame basis. With curve compression, we disable too high jumps between different scenes.
the low/high settings mean, that if a frame is smaller than the average (downscaled already? have to look into the sources), that percentage of bits gets taken away, if bigger than average, the high amount.
Like for nandub, I advise on using 15/25 low/high... have still to see if these values are as nice for XviD as they were for DivX3...)

I hope this helps and isn't too wrong desribed.

Regards,
Koepi

[Franko30]

@ Koepi

THANKS!



Frank

[tangent]

Originally posted by Koepi
Ok, ripping the matrix now to make sure that 1CD still works and looks really nice (well, if quant 2,3,4 get used most often i am SURE it still looks good.).
Just wondering, what's the best XviD setting you've used for The Matrix 1CD?

[Koepi]

You'll find them on the screenshots for the final release of XviD Options Explained 1.0....

Have to d/l and install a new mozilla build though...

Regards,
Koepi

[Ripe73]

Curvecompression

etc low:15% high:25%
Average bitrate:900

A frame with 800 will activate Low% and the new bitrate for the frame will be(800-15%)680.

A frame with 1200 will activate High% and the new bitrate for the frame will be(1200-25%)900.

Is this right
:)

[Koepi]

Nope, you have to think more like:

800 - ((800 * scalefactor) - (800 * scalefactor) * 1-low%)
1200 + ((1200 * scalefactor) - (1200 * scalefactor) * 1+ high%)

e.g.

800 -((800 * 0.8) - (800 * 0.8) * 1-0.15) = 800 - ((800 * 0.8) - (800 * 0.8)* 0.85))) = 800 - ((640)-(640)*0.85)) = 800 - 96 = 704

1200 +((1200 * 0.8)-(1200 * 0.8) * (1+0.25)) = 1200 +((1200*0.8)- (1200*0.8) * 1.25) = 1200 + (- 240) = 960

instead of

800 * 0.8 = 640
1200 * 0.8 = 960
... (scalefactor 0.8 and high% of 25 is a break even point in our example it seems ;) )

Those formulars aren't correct in any way but give you an idea how it works (hopefully).

So huge frames get more bits "taken away" as smaller frames if setup with low% = 15 and high% = 25.

Regards,
Koepi

[Ripe73]

Thanks!!!

But if i dont want to give lower bitrates to frames under the average bitrate i can disable this by Low:0%?

I will test the latest Koepibuild with "modulated"mode and see what happens:)

EDIT:The movie was oversized with 20MB:eek: with the latest Koepibuild but exactly with the old build with same settings

[Koepi]

Hey,

don't blame me, I just adopted the quant_type modulation to the current CVS ;)

Maybe we have to look a little deeper into this. But as I'm on my 2nd pass/matrix now, it seems like the overflow gets distributed correctly...

Must be something with credits I think, I now use fixed quantizer as I think it's the most accurate way to do them...

Regards,
Koepi

[Ripe73]

Must be something with credits I think, I now use fixed quantizer as I think it's the most accurate way to do them...

Well i did not use the endcreditfunction(encoded manual)


Maybe we have to look a little deeper into this. But as I'm on my 2nd pass/matrix now, it seems like the overflow gets distributed correctly...

Well it did to me too(+ - overflow)but in the end there was a huge overflow.

My settings

Blade Runner 167817 frames
2pass Int.
payback:240 C.high:25% C.low:15%
I-frame min:2 max:4
SmoothQ:on
M.Search:5
MainQ min:2 max:6
Qtype:modulated
mostly DRF:2-3

This settings worked fine in your earlier build and now im doing a new second pass with MainQ min:2 max:16 20min left....
The same oversize.

[Koepi]

humm.... and your restrictions weren't unusual high?

like let's say, first pass size 1000000 kb, second pass desired size 600000 kb...
Quant min / max 1-5, I-Quant min/max 1-3, payback delay 385720938560923650 frames ;) , ...

Please, post which binary exactly you used (for example, nic's first binary had some issues, he uploaded a <bla>-part2.zip binary which should correft those results...

"I need more input!" (c)(r)[tm] No. 5 [lives] (sorry, bad joke, very bad movie back from the 80s).

Regards,
Koepi

P.S.: i know you used my build, but I still need your settings :)

[Franko30]

Oh, No. 5 needs input! Thank God No. 5 lives!

So, here it is:
I just encoded an Episode of Star Trek Voyager with your Koepibuild 26022002-1, all standard settings for the new stuff EDIT: except curve compression low15 high 25 EDIT END, quant modulation, "normal" quant settings 1to10 and I frame Quant lock 1-5, quant smotthing enabled, no lumamasking.

Looks very nice - indeed a lot better than the Nic build with symetric curve compression only - I guess the asymetric does the trick. But the blocks on static walls in grey (lots of them in Voyager episodes) are more visible than in the Feb17th build I used so far (with quant modulation). Guess I go back to "h263 only" for 576x432 and a video bitrate of only 935 for that (so that 2 episodes fit on one CD)...

And the resulting filesize was about 8MB over the specified filesize (47 min. clip), no credits enabled - so Ripe73's discovery can be confirmed.

I started my Demolition Man encoding again, with credits (fixed Quant 16) and lots of action (normal quant set to 12) - in the morning I can post more results.

Frank

[Koepi]

My guess is that asymmetric curve compression (or curve compression in general) disturbs size predictability.
Use 0 low/high please and see if this results in the filesize you wanted!

As I wrote down my thoughts about the curve comp I did notice that there is something maybe not right... anyways, I've to look into that :)

Best regards,
thanks for your massive testing :)

Koepi

[rui]

Well, I just did a encoding of the movie Platoon, and I’m sad to report that I too got an oversized file :(
My settings were for first pass H.263, 5 motion search precision, and I enabled the end credits, choosing for the credits a 31 quantizer.
For the second pass I chose 5 motion search precision, modulated quantization, quant settings 1-31, I-frame locking 1-5 with the smooth parameter enabled, payback:240 C.high:25% C.low:15%.
I used the latest Koepi's build.

The quality was unbelievable good :) (this is a 1 cd rip), but I targeted for a file size of 614633 and got 628526 :(

Other than that, no problems whatsoever.

[sierrafoxtrot]

more results ...

battle royale
statsfile generated with nic's 25/02 build
motion search 5
modulated quants
intra lock 1 4 smoth quant enabled
inter lock 1 6
asymmetric curve correction low/hi 17/25 payback delay 250
no luma masking
credits quant 20 (found that % bitrate setting gives me overflows but i haven't used it since the new overflow correction was implemented)

desired size: 619670
final size: 629320

whups ;) (but i got to agree with rui ... the quality is superb!)

anyway, will run another overnight with disabled asymmetric correction. and probably another with external 2-pass (haven't seen anyone else try this out ...)

pity about the size overshoots guys ... but i can verify that symmetric correction works perfectly. encoded battle royale, and josie and the pussycats and both were within 150kB of desired filesize.

keep those builds coming! :D

[Koepi]

I am at the credits as well now and debugview shows:

[936] 2nd-pass: quant:20 type:.h263 inter stats1:142 scaled:14474 actual:142 overflow:7877195 credits

so there is some error that the credits still get scaled while they shouldn't be....

Just to give some feedback...

Regards,
Koepi

[Nic]

(After hours of trying to get the DivX4a50 sources to work correctly, I realised that I had assumed the decore on videocoding.de was equal to the divx4a50 source...this is not the case.....ARGHHHHHH! (Sorry just had to announce my frustration :)