My XVid binaries

[rui]

Nic, the dll file is the same size than Koepi's dll.
This means that you are back to VC 6?
Any problems with VC 7?

[Koepi]

Erm... I'm not in a battle here. I'm trying to do some extra coding on XviD to improve it - so it has nothing to do with the usual binaries. And I may not do this for long.

*grrrrr*

All my releases have some "extra features" that might break something, result in unpredictable behaviour, is unusable for very low bitrates etc etc.

I won't upload my actual binary for that reason, just to show that I'm not willing to be in a combat.

Just a hint:

The test with modulating the quant_type on a per-frame base (dependent on the quantizer) works like charme and results in a sharper overall movie. But it triggers the DivX4-DS-Filter playback bug giving some green areas (very seldom, but they are there).

Best regards,
Koepi

[saVe]

is there no way you could make up your mind? i'd love to test your "special" binaries..... please?

[Nic]

@Koepi:
Oh, theres no battle, your doing far more than me...Keep up the good work, I only started to do it, because uManiac was updating too slowly....& because its very easy.

@Rui:
I did go back to VC6 your right (very sharp :) I did a very brief test on a SVCD of Lord of the Rings....I was getting about 26fps with VC6 & about 25 with VC7, I think???? Im not sure...There definitely isn't any huge speed increase....However there are some settings I could play around with in VC7 to make it that little bit faster.
Seeing that most of the important bits of the codec are ASM the compiler cant speed that up. So for now VC6, unless anyone noticed improvements when using VC7....In which case ill go back to 7.

(I should try compiling DVD2AVI & Lame now with VC7 :)

-Nic

[-h]

Erm... I'm not in a battle here. I'm trying to do some extra coding on XviD to improve it - so it has nothing to do with the usual binaries. And I may not do this for long.

Ominous words? BTW the extra debug output that you added is in the CVS, just uncomment the junk after DEBUG2P(), DEBUG1ST() and DEBUG2ND() in codec.h, and put it on the same line as the respective #define. I remember reading that you thought it was only enabled in CVS Debugg builds?

The test with modulating the quant_type on a per-frame base (dependent on the quantizer) works like charme and results in a sharper overall movie. But it triggers the DivX4-DS-Filter playback bug giving some green areas (very seldom, but they are there).

Yeah they're dragging their feet when it comes to fixing the MPEG quant issue. Just wondering, what threshold did you use to switch from MPEG to H.263, and back again?

-h

[rui]

Originally posted by -h


Yeah they're dragging their feet when it comes to fixing the MPEG quant issue
-h

Arghh!!

Those divx4 guys... :rolleyes:

Just wondering.. It's still far the time when XviD will have its own post processing filter? Then we wouldn't need them anymore.

[-h]

Just wondering.. It's still far the time when XviD will have its own post processing filter? Then we wouldn't need them anymore.

Ehh....

If I have some spare time this weekend, I'll look at duct-taping ffmpeg's post-processor onto XviD. I've never used it before, nor do I know if it's any good whatsoever, but the source is there to play with.

-h

[Koepi]

Originally posted by -h

Ominous words? BTW the extra debug output that you added is in the CVS, just uncomment the junk after DEBUG2P(), DEBUG1ST() and DEBUG2ND() in codec.h, and put it on the same line as the respective #define. I remember reading that you thought it was only enabled in CVS Debugg builds?

Uh, stupid me! I just tried commenting that out without moving it on the same line... well, I'm still no pro when it comes to coding ;)

Yeah they're dragging their feet when it comes to fixing the MPEG quant issue. Just wondering, what threshold did you use to switch from MPEG to H.263, and back again?

I hardcoded using MPEG4 quant_type for quantizers 1-3, .h263 for 4-31.



Best Regards,
Koepi

[rui]

Koepi, i too would like to try your builds. The alternating quantizer seems a good ideia, at least to me. (i am no coder). Then we hould have the best from both worlds.
Your site clearly (is this correct?) states that the build is experimental, so anyone that tries it should accept any little bugs.
Your work is very apreciated, be sure of this.

Hope -h can get that post processing working ;)

[Koepi]

I made my stand, to show that I'm not willing to "fight" or anything I'll post no release today.
There's something left to do which improves the release a (very) little, something you could describe as "dictating the core to use I or P frame at our will".
With that modification there won't be any I-frames out of the lock range.

I may post that resulting binary tomorrow or so - but nothing today!

Best regards,
Koepi

[Nic]

:)

I think people know theres no fight or anything Koepi....Why would there be? :)

Im sure people will look forward to it tomorrow :)

Take Care,
-Nic

[ChristianHJW]

Originally posted by Nic Hmmmm...The new project manager at work has very good DShow experience....maybe I can rope him in as well :) :)Nic
.... yeah, good idea, drag him here :D

[ChristianHJW]

Originally posted by Koepi Erm... I'm not in a battle here. I'm trying to do some extra coding on XviD to improve it - so it has nothing to do with the usual binaries. And I may not do this for long.
Sorry mate, i didnt want to step on anybodies feet, definitely not my intention. I was basically posting my comment because i thought it could help to win Nic for the MCF parser project :D .... but he decided to help my anyway and couldnt get through to me because my inbox here is going crazy ( ?? ) ..... never mind, keep up the excellent work on XviD my friend !!

[Ripe73]

HI Again!!!

Encoded a movie with XVIDCVS 150202
132678 frames
2pass H:263
L.mask second pass
M.Search 5
I-Frame Min:1 Max:4
Smooth Quantizer
The movie was 3MB oversized:D

Did a new second pass with I-frame Min:1 Max:6
the rest samme settings
perfect size:)

The Debugview did not work with this build for me. :mad:

Thanks

[Koepi]

As I wrote earlier, I-frame quantizer lock can produce oversized files when given to strong tresholds.

So, nothing to be surprised of :P

Regards,
Koepi

P.S.: the debug output is disabled by default in CVS.

[-h]

As I wrote earlier, I-frame quantizer lock can produce oversized files when given to strong tresholds.

I haven't actually encoded anything over 2 minutes in length with XviD - are long movies uniformly oversized by 2 or 3 MB? I have always wondered about the 2-pass code not taking into account AVI's overhead per frame - over a long movie, this has the chance to add up to a few MB oversizing, and it'd be easy to just add the AVI overhead (~24 bytes per frame?) to the 1st pass frame size before it's written to the .stats to fix this.

Then again, if no uniform oversizing is occurring... I'm just confused :)

And yes debug output is disabled by default in the CVS. My next commit will enable the 2-pass-specific debug output (you don't want to see everything outputted by XviD..)

-h

[Koepi]

Hm. I tried a single movie 3 times, targeted ~610mb, got always the _same_ size, ~613mb.
We might try to add 24 bytes per frame, the movie had 122000 pictures, resulting in...
2.79MB overhead.
would explain the matter perfectly!
But I would simply use that on second pass, not for the stats file, but in overflow += frame->length + 24 ;)

Best regards,
Koepi

[Foo]

Originally posted by -h
I haven't actually encoded anything over 2 minutes in length with XviD - are long movies uniformly oversized by 2 or 3 MB?



Yes :). I've done several full length features, both one disc and two. Each one is just *slightly* oversized. I actually came here tonight to ask about this...nice timing.


I have always wondered about the 2-pass code not taking into account AVI's overhead per frame - over a long movie, this has the chance to add up to a few MB oversizing, and it'd be easy to just add the AVI overhead (~24 bytes per frame?) to the 1st pass frame size before it's written to the .stats to fix this.


Please do! Thanks for all the hard work everyone (-h, Nic, Koepi, etc). This would just be one more nice little thing about Xvid. Right now I target at 698 to be safe.


foo

[Koepi]

@-h:

Morning,

I just did a few modifications to codec.c, may you have a look at this?

in "int codec_get_frame_quant"

I extended "frame->intra = -1" to


if ((codec->config.mode == DLG_MODE_2PASS_2_INT) || (codec->config.mode == DLG_MODE_2PASS_2_EXT))
{
frame->intra = (nns1->quant & NNSTATS_KEYFRAME);
}
else
{
frame->intra = -1;
};


and (just a temporal workaround since my stats file is already done)


case DLG_MODE_2PASS_2_INT :
case DLG_MODE_2PASS_2_EXT :
// Modified by Koepi
codec->overflow += bytes2 - frame->length -24;


added a -24 (AVI frame overhead) in "int codec_2pass_update".

What do you think about these modifications, are they correct?

Regards,
Koepi

[Koepi]

Well, there were no objections so I uploaded new binaries.
Find them at http://roeder.goe.net/~koepi/ .

Some of the changes:

- Quant type modulation implemented, you have to select that on second pass in the config GUI (quant type: modulated).
- Added the AVI header size to overflow on 2nd pass internal. File size should now be precise to some kb, if the config settings aren't too strict.
- Frames that were smallest size possible (around 101 bytes) don't get scaled anymore (well, 101 bytes hardcoded now, have to figure out how to find the treshold).
...
and some things I forgot.

Modifications to the sources are included in the archive as usual.

Best regards,
Koepi

[Demone]

I encoded Aliens with Koepy's build (not the last one) and I cant tell the quality in % but I think its something near 60-70% at 576*xxx
in 3cds with dolby. I must say that I cant notice pink green and other shit, but there is something strange...when there is a scene change u actually can notice the keyframe, I mean there is a jump in bitrate that u can see in picture quality and after (quite fast) the quality deteriorate, mantaining a constant (but not good as the keyframe) quality. It seems to me that or the decoder is not working as it should or someone must put hand on the i-p-b frame relation.

I got and AthlonXP 1900 + K7V266-e geforceMX +latest
I did a two pass with the second internal, giving 1718000 as final size using mpeg-quant 300maxkf and no lumi in either 1st or second pass.

[Koepi]

It is important to know WHICH build you used.
This phenomenon you describes sounds like iframe quant lock without quantizer aliasing.

Regards.
Koepi

[Demone]

Its the 10022002 but was umaniacs...sorry koepy going to try ur latest.

[Koepi]

I made the attached graph below to explain what quantizer aliasing does (and what it can't do...).

See those intraframes (red). Those got caught by the iframe quantizer lock, as you can easy tell from the frame behind, which is smaller than the usual ones (right after that).

With the older builds, there was no quant aliasing and the frame right after such an intraframe got _very_ small resulting in a ugly frame/image.

now you can see, the frame still gets smaller, but the negative overflow is distributed over some frames (the other yellow arrows show this. There is a bigger frame missing where "usually" there would have been some).

This looks way better when watching the movie.

But I wrote already earlier, that this iframe quant lock and quantizer aliasing an't do wonders. It won't look nice if you set the values too strict as the p-frames get smaller (a higher quantizer is used) resulting in an overall nice, but strange look of the images.

I hope this helps explaining that phenomenon a little.

Best regards,
Koepi

[Franko30]

Originally posted by Koepi
Well, there were no objections so I uploaded new binaries.

- Quant type modulation implemented, you have to select that on second pass in the config GUI (quant type: modulated).
Best regards,
Koepi


Hi Koepi,

I just installed the new binaries and while encoding my first test file, I was wondering why Quantizer modulation should only be used on 2nd pass. I guess there's an obvious reason I just don't get the hang of right now...

Frank

[Koepi]

You need the statistical data on first pass on a constant DRF, that's why. The same reason applies to luma masking :)

The scaling on second pass would go wrong if you use that on first pass, resulting in a very ugly video...

Regards,
Koepi

[Franko30]

@ koepi:

Thanks!

I knew it was s.th. like that, just couldn't put it to words.

Another question:

Some people say that the MPEG quantization brings the effect where, on non-moving areas of one colour, the surfaces seem to be moving.

Did you notice that, too? Not that it matters much anyway, with my analogue sources...

Frank

[rui]

Originally posted by Koepi
The test with modulating the quant_type on a per-frame base (dependent on the quantizer) works like charme and results in a sharper overall movie. But it triggers the DivX4-DS-Filter playback bug giving some green areas (very seldom, but they are there).

Koepi, couldn't this be avoid by selecting XviD fourCC, in the codec tab? Or is the quantizer modulation for itself that triggers divx4 post processing even if we do that?
And regarding the question Franko30 made about the quantizer modulation being choose in the 2 pass:
in the first pass we can choose either H.263 or MPEG?

Thanks

[Koepi]

hm. in the first pass you can choose whatever you want (but don't choose modulated as that would result in MPEG quant type ;) ).
I use .h263 there, because most of my frames will use that quant_type later on. But you can try as well using MPEG, maybe this gives better results, mybe worse, maybe the same :) (if I understood it correctly, MPEG quant_type frames are bigger at higher quantizers, so the frames get scaled to big on second pass... it finally shoudl depend on the resulting average quantizer then. If you have more 2x and 3x frames, you should use MPEG on first pass, but if you have more frames >3x you should use .h263)

And nope, those modifications don't need the DivX4 DS filter, it's just that, if you use FourCC DIVX, that bug can get triggered. Everything should be fine with the XVID FourCC.

Regards,
Koepi

[serbersan]

I just encoded a movie with the 15 fb build of Nic and appears some macrobloks like freeze frames in divx3 several times along the whole movie but now I don't have antishit.

The effect isn't the same as freeze frames but very similar. The colours of them are strange red, pink, green, blue, pure colours.

In fact in one scene I've noticed the bottom of the screen with those colours merged during 3 or4 seconds.

I encoded the movie using the default values and using luma masking in 2º pass.

I've noticed too, black horizontal bars when the screen is absolutely in black.

Without this the overall quality is very good.

[MaTTeR]

Originally posted by serbersan
I just encoded a movie with the 15 fb build of Nic and appears some macrobloks ...

The effect isn't the same as freeze frames but very similar. The colours of them are strange red, pink, green, blue, pure colours.

In fact in one scene I've noticed the bottom of the screen with those colours merged during 3 or4 seconds.

I encoded the movie using the default values and using luma masking in 2º pass.


I can verify this as well. I used the same settings and got the same results. Try the latest build from Koepi now:)

[-h]

Well, there were no objections so I uploaded new binaries.
Find them at http://roeder.goe.net/~koepi/ .

Oops, haven't been online for a while.

- Frames that were smallest size possible (around 101 bytes) don't get scaled anymore (well, 101 bytes hardcoded now, have to figure out how to find the treshold).

How often does this happen? The smallest possible frame size (with current builds) is (Vopheader + 1 bit per macroblock + markers). Or it should be, anyway..

In terms of frame size for H.263 and MPEG quantization, the relationship seems to be fairly linear. If I get around to coding a replacement for the frame.quant decision, I'll try implementing it and refining the 2-pass mode further when quant modulation is enabled.

Tomorrow should be a big day.

-h

[Koepi]

Hehe, fine then :)

My GF is away for ~48hours from now, so I can spend some more time on the computer without getting into trouble, so it would be nice if we could use that time :)

Regards,
Koepi

[Koepi]

As I'm currently in the last minutes of encoding my movie, I see one thing we must change as well:

we should lift the iframe quant lock if credits are active... debugview output is REALLY ugly at the moment ;)

Regards,
Koepi

[Koepi]

Hi again,

now that the movie is finsihed, I wanted to post my results:

The quality is unbelievable. It is impossible to achieve with DivX3/Nandub IMHO (with these size limits etc).
My desired size was 569.6875MB, it came out 569.7109MB (so just a thing caused by iframe quantizer lock in the last few frames). So it's definatly been the 24 bytes avi-header-overhead per frame which gave a little size unpredictability!

Btw., the movie is The World Is Not Enough, 2 hours 1 minute long. I'll mux 2 soundtracks (vorbis, qual 0.01) with it into 1 ogg and hopefully it hits 700MB... :)

So far my results,
Regards,
Koepi

[Synth]

What res what the encode at?

[Koepi]

640x272.

As you would expect, sometimes large unicolour areas are a little blocky, but hey, I now have a bilingal movie of 2 hours in really NICE quality on 1 cd.

:)

[ChristianHJW]

@koepi

What was the average bitrate of your Vorbis audio streams at q 0.01 ?

[Demone]

I tried ur latest build and I'm comparing it with 3.11, for backgrounds xvid is better, but it seems that the 3.11 has a finer detail, seems that xvid is more brutal as image quality, I mean grainy, but I'll do more tests.
Can u put up some seconds of your encoded movie ?

[Ripe73]

HI!
Use XviD-16022002-1 and when i watch the Debugview i see alot of double keyframes and sometimes more like 5-6 keyframes(in a row) .Whats wrong?

EDIT
[3524] [XviD-2nd] Quant:2 .h263 INTRA Stats1:31079 ScaledTo:23324 Actual:31079 Overflow:-21920
[3524] [XviD-2nd] Quant:3 .h263 Inter Stats1:31055 ScaledTo:23306 Actual:21472 Overflow:-20110
[3524] [XviD-2nd] Quant:2 .h263 INTRA Stats1:29100 ScaledTo:21838 Actual:29100 Overflow:-27396
[3524] [XviD-2nd] Quant:2 .h263 INTRA Stats1:30710 ScaledTo:23047 Actual:30710 Overflow:-35083
[3524] [XviD-2nd] Quant:2 .h263 INTRA Stats1:30710 ScaledTo:23047 Actual:30710 Overflow:-35083
[3524] [XviD-2nd] Quant:5 .h263 Inter Stats1:29614 ScaledTo:22224 Actual:12368 Overflow:-32887

EDIT AGAIN :D
When i jump between keyframes in V-dub i cant see any (K) on these framenumber Debugview showed as keyframes:rolleyes:

[Synth]

Im using Koepi's 16-02-2002 build.

What do you suggest for min and max iframe quant?

What impact (visually) does 'smooth quantizer fluctuations' have? In other words, whats the benefit of this function?

Is the only limitation of using the xvid decoder (as opposed to using the div one) the fact it cannot post process?

If indeed the only limitation is that, for HQ encodes where enough bits are available wouldn't it better to have the xvid do

the decoding?

Lastly, for purpose of burning onto cd / future playback...Do you suggest I burn with xvid set as the decoder, so that in the

future when it does get post processing abilities I could benefit without having to re-burn and change the header from div to

xvid, or will the issue (green & pink bugs) with div 4 decoder be resolved eventually?

[Acaila]

ChristianHJW wrote:
@koepi

What was the average bitrate of your Vorbis audio streams at q 0.01 ?

I'm not Koepi, but I just wanted to respond anyways :)

At quality 0.01 I always get 64kbps for 2ch audio and 192kbps for 6ch audio. Is this normal?
Like if I wanted a 6ch stream at for instance 128kbps it's not possible?

[Koepi]

That's the nominal rate, this is lowest: 32 kbps per channel.
2 channels result in 64kbps nominal, 6 channels in 192kbps nominal (6 channels has no channel coupling btw...).

I got (of course) 64kbps nominal, german audio was 63 kbps real, english 62...

Regards,
Koepi

[MaTTeR]

@Koepi

With your new build (XviD-16022002-1) I'm still seeing the nasty green bugs. I was thinking this bug had been fixed.

Auto Optimizations for PIII
Gknot 2ps with Luma-Masking
Used both WiMP, PowerDivX 4a7 and PD4a8.

The artifact is mostly green but I do see other colored blocks as well. Let me know if you need more information.

Thx,
MaTTeR

EDIT- I should add I used the default encoder setting motion search(5),H.263 and FourCC=XviD.

[-h]

What do you suggest for min and max iframe quant?

I'd say 2 to 3, for a 2-CD rip. Or 2 to 5 otherwise.

What impact (visually) does 'smooth quantizer fluctuations' have? In other words, whats the benefit of this function?

When you change the I-frame quantization settings from 1 to 31, the codec is no longer able to assign an I-frame exactly the quantizer that it wants to. Because we're overriding the 2-pass code (and usually making I-frames much larger by doing so), the frames immediately following the larger I-frame will be quantized heavily to keep the desired size in check. Since such quantization jumps look pretty ugly, Koepi came up with the idea to restrict jumps in subsequent frame quantizers to +- 2.

Is the only limitation of using the xvid decoder (as opposed to using the div one) the fact it cannot post process?

Yes.

If indeed the only limitation is that, for HQ encodes where enough bits are available wouldn't it better to have the xvid do the decoding?

Not at all.

Lastly, for purpose of burning onto cd / future playback...Do you suggest I burn with xvid set as the decoder, so that in the future when it does get post processing abilities I could benefit without having to re-burn and change the header from div to xvid, or will the issue (green & pink bugs) with div 4 decoder be resolved eventually?

In the future, you'll be able to play avi's that have the DIVX FourCC using the XviD decoder, so it won't really matter too much. Depends on preference I guess.

-h

[Synth]

@ -h: Thanks.

@ Matter: Using the XviD-16022002-1, I get pink bugs (okay more like purple) with my athlon and motion search=6, Quant=MPEG, NO lumi.

But it goes away when using the Xvid as the decoder.

I was so sure it was a strictly divx decoder fault...Now maybe I think its also a lumi issue? In anycase isn't the lumi function still much a work in progress, more so than the rest of the codec? Thats why I'm staying away from it for now. Plus my encodes look better without the lumi enabled.

[Synth]

What about the user being able to choose the level of "smooth quantizer fluctuations" instead of the fixed +/-2?

[MaTTeR]

@Synth

Thanks for the feedback. Maybe your right and it is the Luminance Masking. I'm going to rerun my 2nd pass with it set to off and try again.

I'm using the XviD as the decoder and still seeing the issue. Is there a way to ensure I'm using XviD? I thought I seen mention of a utility that would change the FourCC code.

EDIT- I gotta say that the overall quality of this latest build is simply superb. I hardly see any macroblocks in the shadows:)

[Synth]

I just tried WITH lumi...but xvid decoder..no bug.Course its entirely possible it could be the specific source/scene itself that helps initiates this bug or a combination of factors.

Its only a matter of time before we narrow down the exact reason and fix it.

I dare say using 2CDs with this latest build gives quality almost indistinguishable from actual dvds.

FourCC changer here:
http://www.inmatrix.com/

BTW Im also using iframes 1-3 and 'smooth quantizer fluctuations'.

/me remembers the many weeks of encoding time spent trying to get the holy grail of SBC settings in his youthful days :D

[Synth]

Just a random idea...check the "About" section in the codec..check if the date is the 16th.