XviD-12072002

[Koepi]

Hi,

I did it again...

Based on CVS from 12.07.2002 15:00h MET...

Changelog:

XviD-12072002-1:
- EPSZ(^2) Motion Estimation activated
- BFrames activated - but they buggy and ugly, be warned! Don't work with MV hints.
- Compiled as SMP version (for multiprocessor usage)

Find the thread number on the debug-tab.
B-Frames are highly experimental and known to look ugly - so no reports about this here! They should at least work in 2pass mode now as well (but only correctly when no other option except the "debug print info on video frame" and "DX50 BVop compatibility" are checked. Packed bitstream confuses the curve treatment algos and leads to unpredictable results. For playback you should use ffdshow, the XviD DSF included in the package doesn't seem to decode the bframes.)

Well, after all these warnings, enjoy the new build and educate yourself and us! :)

Regards,
Koepi

[rui]

Ok boss, going to give it a shoot.

I promisse not to complain :D

[Acaila]

Thanks for the new build Koepi :)

I did some testing since I was very eager to try out the new SMP code.

I set B-frames on -1 so that couldn't cause any slowdown. Test clip was 10.000 frames long, quantizer type MPEG, 2-pass mode to maximum quality so size would be comparable to 1-pass 100% quality mode. Also tested the effect of Lumi Masking. And lastly I also ran the same clip with a TempSmooth(1) because that drops average cpu load to approx 70%, so more room for SMP effect compared to a normal level of approx 90% average load.

Results
Build Lumi Mode Time (sec) Size Speed (fps)
22-5-2002 No 1-pass quality 100% 187 67128 53
22-5-2002 Yes 1-pass quality 100% 188 65574 53
22-5-2002 No 2-pass 1st 189 -- 53
2-pass 2nd 188 67126 53
22-5-2002 No 1-pass quality 100%, TempSmooth(1) 390 60430 26

12-7-2002 No 1-pass quality 100% 193 66980 52
12-7-2002 Yes 1-pass quality 100% 193 66920 52
12-7-2002 No 2-pass 1st 195 -- 51
2-pass 2nd 196 66986 51
12-7-2002 No 1-pass quality 100%, TempSmooth(1) 413 60304 24

As you can see there was absolutely no speed advantage caused by the SMP optimization. I was quite dissappointed in seeing this, I know the effect would be very small, or so gruel said, but I didn't expect nothing at all.

The new Lumi Masking code still does not provide any compression advantage. I haven't heard about development in that area a while so that was to be expected. Can anyone tell me if someone is working on that?

I didn't test out the B-frames yet, this was mainly about SMP for me.

But anyway, it's always great to have a new build :D.

[spyder]

Koepi:

I'm working on this XviD wrapper in Java and I would really appreciate it if you could post a core.lib that you compiled. Mine seem to be very buggy, maybe compilation problems.

BTW: I'm using VC6 if it matters.

Thanks,
John

[NeVeRLiFt]

woot! woot!

Will test the B-frames asap :D

Will also do some videocapture using this version and use the B-frames, FYI this really seems a good way to test B-frames I know it works for DivX5 and after much tweaking I actually have it so DivX5 can be used for videocapture ;) using the B-frames and some other settings. (just testing and tinkering)

[Selur]

*sniiif* :( can't get on koepi's site :( *sniiif*
(seems like it's down atm, will try later,...)

[Vanos_b]

Selur is right. I tried last night and this morning too, but the site cannot be found. Koepi, would you mind checking it out?

[spacer]

can somebody post the exact filename?

so i can search 4 a mirror & get it an my webspace or even on an FTP server 4 all of you

by the way, @koepi, CHANGE THIS (http://forum.doom9.org/showthread.php?s=&threadid=24358) ;)

[Koepi]

I'm sorry mates, somewhere among our route into the net there was a machine down :-/

It lasted from yesterday, 21h local time, until 14:25h today local time (that's exactly 5 minutes that we're on the net again...)

Regards,
Koepi

[Acaila]

@Koepi:

Using this release I have noticed that the codec no longer maintains the minimum I-frame spacing like it should. As always I have it set on 5 and now sometimes I get 2, 3 or 4 intra frames in a row.

Do you have any idea what might be causing this?

[Koepi]

Acaila:

I noticed that, too.

Didn't have the time to check that, I'll do later on today, just came back from shopping - and now I've to cook something first, I'm kinda starving ;)

Any other bugs found already? I noticed something that could be described as luma-inverted colour block like in good'ole nandub with a test encoding I made tonight, can someone confirm this? (BFrames used, seems to happen on high quantizer levels)...


Regards,

Koepi

[kastro68]

Koepi's 12072002 build does not seem to work for me at all.

The B-frames do not work and the size is way off target. Instead of getting a target of about 200 megs, I got something that was of the magnitude of 400 megs.

The encoded video was also very shaky and nervous.

Does Koepi's build work for anyone else?

[Koepi]

kastro,

maybe you should reset the values (you know, that "defaults" button in the config-gui) and try again.

It obviously does work for other people.

Maybe you didn't respect the warnings I wrote: no MV hints,...
It's known that b-frames are buggy, and if you use them, you're on your own :)
(BFrames are incomplete yet. The curve gets scaled wrong on 2pass modes. And so on...)

Disable B_Frames by setting "max b-frames" to -1 again, and then you can have trouble with the multithreading... well, but this "release" is just for testing those features. XviD is alpha[tm]! :)

Regards,
Koepi

[kastro68]

Hitting the "Load Defaults..." button appears to have fixed my problem.

Thanks.

[kastro68]

I was re-encoding a VCD clip using the 12072002 build from Koepi and noticed some bleeding from the the bottom edge. It bled throughout the entire 30 min clip.

I think it was because I cropped the black borders off the VCD clip.
Original resoln:352x240
Cropped resoln:348x228
Is it required for me to keep the resolution a multiple of 16?


I also found that the file was about 20 megs undersized, but I believe this is because the clip is saturated at the current bitrate. Reducing the B-frame compression would probably solve this issue.


Edit:

I think I should also mention that:

-I was using Mpeg quantization for first pass and Modulated for
second pass.

-I was using DX50 FourCC

-DX50 B-VOP compat. was ticked

[Nasse]

if i remember right u should use h.263 in the first pass when using modulated...

[Koepi]

It shouldn't matter much which quantizer type you use for the first pass, but I still recommand using h263 quant.

And yes, your image size has to be multiple of 16, else you get funky results...

Regards,
Koepi

[Acaila]

Actually it depends on the movie. On the 1st pass you will want to use that quantizer type that gets used most on the 2nd pass. So if you would end up with an average quant of anything under say 3.5 you will want to use MPEG quant on 1st pass to make your encoding as efficient as possible.
You can actually see the difference of H.263 or MPEG on 1st pass on the final quantizer average of the second pass.

[libredr]

Koepi, could you leave on your site the previous version of Xvid (the 30062002 version) ? I would like to compare both, but I did not download this one (holidays...)
Thanks.

[rui]

Well, i have been doing a number of tests using the latest Koepi’s xvid build and enabling b-frames in 2-pass mode, and come to the following conclusion:

In slow motion scenes, b-frames work great. By doing a visual comparing between a clip encoded using b-frames and another not using b-frames, in the slow motion scenes, the b-frames one is noticeable more detailed. So, great correct?

Not quite.

When the scenes get fast motion, the b-frames build sometimes still has some problems displaying the scenes correctly. It’s noticeable that with b-frames disabled the resulting video is more perfect than when using b-frames (in some, not all, fast motion scenes, remember).
But I must mention that I still didn’t have done a full movie test using b-frames. The larger I have done was a 40 minutes clip from the movie “The Haunting”. And, when choosing the final size, I have tried to reproduce the kind of bitrate that one would have to use in a 1 cd encode (700-900 kits/s).
In some tests, I noticed that when increasing the bitrate, the fast motion problems that b-frames have weren’t so noticeable (and maintaining the superior image detail in comparison with the non b-frames encode).
But the purpose of b-frames is giving us the possibility of doing better 1 cd encodings, correct? So in the majority of the encodings the bitrate will be small.

But I believe that when these little quirks have been solved, we have a winner!

My settings were:

All codec 2-pass defaults, except:

Motion Search Precision 6;
1st pass quantizer-H263; 2nd pass quantizer – Modulated
Lumi enabled in both passes;
Maximum B-frames-3
B-frame quantizer ratio (200%)
DX50 B-VOP compatibility enabled.
Default CC with: Hi bit –20; Low bit-10; delay-300;
No HintME, of course;

I got the b-frames values from reading some posts both here and in the xvid.org forum. I don’t know if they are the best ones, please someone correct me if I have done something stupid with the values.

[Acaila]

I agree that B-frames don't look so well on fast motion scenes (didn't gruel comment on that on xvid.org? not sure...).

I have also noticed that the B-frame quantizer ratio only has an effect when used with 'round' numbers, with effect I mean a size decrease when tested on 1-pass 100% quality mode. I.e. 100% works, 150% works, 200% works, 250% works, 300% works, etc. Anything in between those values (e.g. 175%) has no added effect over the matching ratio right before it (175% = 150%).

Can anyone confirm this?

[rui]

Originally posted by Acaila
I agree that B-frames don't look so well on fast motion scenes (didn't gruel comment on that on xvid.org? not sure...).

Yes, he did. Sorry for not mentioning that :(

quote:

Question from TactX: "In high-motion parts, B-frames are less useful then in low-motion parts, right ?"

Answer from gruel:

"During high motion, Motion estimation is less effective, and the interval between P-frames gets more important, so the answer may be "yes". "

But i believe that once b-frames are fully developed, they will not be damaging for the final video in fast motion scenes.
Note that the answer NO was to the fact that b-frames would be less usefull in fast motion scenes, and not damaging.

The discussion about b-frames in xvid.org is here:http://www.xvid.org/modules.php?op=modload&name=phpBB2&file=viewtopic&t=655

[Bulletproof]

If it's damaging to high motion scenes why not just determine when to use B-frames by looking at the motion curve and only use IP frames for those segments. Or does that violate the GOP?

[kastro68]

Did you do a comparison with the DivX5 codec? Would the divx5 codec also perform as poor in high motion scenes?

And how was the size estimation of your encoded Xvid clip with B-frames?


Ja

[rui]

Hi kastro68

No, i didn't made a comparison between the 2 codecs because i stop using divx5 a long time ago. Nothing against it :D, just decided to stick with xvid.
So, all my expericences don't mean that xvid is superior/inferior to divx5, it's only my take on this matters.
Regarding size estimation, all my tests (2-pass) were always spot on, give or take a few kb.


Bulletproof: maybe my words weren't carefully chosen. In my experiences, the b-frames enabled xvid build wasn't so good in fast motion scenes as the non b-frames build (well it's the same build, only one encode has b-frames enabled and the other doesn't). Maybe damaging is a too strong word.

But, like i said, i didn't made a full movie test, and if i raise the bitrate some, the fast motion scenes didn't look bad. Maybe in a full movie, the codec has more, how should i put it, "movie" to distribute the bits better, and the fast motion scenes would look also good.
I must test more.

But it's so freakin hot here in Portugal, that i don't want to even turn on my computer...

[kilg0r3]

the motion-curve switch suggested by bulletproof seems to be a good idea.

[Nic]

But it's so freakin hot here in Portugal, that i don't want to even turn on my computer...

Thats what I like to hear :)...just hope it stays that way until the 27th & beyond :)

-Nic

[rui]

Originally posted by Nic


Thats what I like to hear :)...just hope it stays that way until the 27th & beyond :)

-Nic

:D :D Well, i am sure that in Algarve it will.
By the way, in the local news they are saying that this year Algarve has less people, and some hotels are making some bargains to get closer to full capacity. ;)

Ah, how i miss those days when i was only a student...and had 3 months of vacations... :D

Sorry to the mods for the off topic post.

[Neo Neko]

Wow! Well B frames need work. And there are some bugs and stability issues. But I have done test encodes and must say I am impressed. Massive size reductions with some small quality reductions. Comparing to a Divx 5.02 Pro encode of similar settings the Divx 5.02 does B frames better. But not by much. But this is one of the first few binary releases of Xvid out with B frame support. It is all still very experimental and new. But I can't describe to you my anticipation of the next B frame update. :) Keep em comming guys!

[rui]

Well, i just saw at uManiac's site this info, related with a new compile from the CVS:

U xvidcore/src/encoder.c (rev.1.56) chl:
- Fixed for Bframes encoding and maybe decoding, too.
U xvidcore/src/bitstream/bitstream.c (rev.1.24) chl:
- Fixed for Bframes encoding and maybe decoding, too.
U xvidcore/src/bitstream/mbcoding.c (rev.1.21) chl:
- Fixed for Bframes encoding and maybe decoding, too.


Maybe this is worth a new compile from Nic/Koepi's ?
I don't want to be a pain in the ass asking for new builds, just that maybe this means that xvid is now a lot better with b-frames related?

[rui]

Originally posted by Acaila
I have also noticed that the B-frame quantizer ratio only has an effect when used with 'round' numbers, with effect I mean a size decrease when tested on 1-pass 100% quality mode. I.e. 100% works, 150% works, 200% works, 250% works, 300% works, etc. Anything in between those values (e.g. 175%) has no added effect over the matching ratio right before it (175% = 150%).

Can anyone confirm this?

I maybe can. :)

I made 4 small encode tests, using your configs (all default quality 100% mode), but changing the b-frame quantizer ratio from 150; 175; 200; 225.

The 150 and 175 had EXACTLY the same file size: 72.558 KB.
The 200 and 225 had also the same file size: 32.058 KB.

So, this kind of confirms your findings :)

This only proves that you are a sharp fellow :D

[Acaila]

So, this kind of confirms your findings :)
This only proves that you are a sharp fellow :DThank you. For both those comments :D

[rui]

And i made another test.

After looking with more attention to the results, i saw this:

- the 150 and 175 encodes had the same size.

But at what point would the encode, lets say the 150 and 175, get smaller in size?
Maybe breaking the 175 mark, the encode would get the size of 200?

So i made two new ones, the 1st at 176 and the 2nd at 199.
The 176 one had the same size that the 150 and 175. And the 199 one also had the same size of the 150, 175, and 176.

Conclusion: after passing the mark of 150, all the encodes remain equal in size until the next 50 points are reached (200). Then the size gets samller, but remains equal again until the next 50 points up are reached (250), and so on.

So, it isn't of any use to change the ratio to other values outside the 50 points interval, like 150; 200; 250; 300..., like Acaila first found out.