XviD-1.1.-127-13102004...

[Koepi]

Hello mates,

since I had a few testers who did really hard work on testing the build I decided to put it online now for public download. Find it over at my XviD page at www.koepi.org !

I want to thank especially Sharktooth for his graphs and PSNR tests, as well as yaz and omni.

Enjoy (and don't forget it's unstable, for testing features...)!

Regards
Koepi

[Sharktooth]

B-VHQ increases quality of b-frames in almost every case (and slowes down the encode job).
VBV buffer "should" work (at least i tried with only 1 standalone...) and image quality impact is imperceptible.
Loose Curve Scaling is similar to Xvid 1.02 curve scaling, while Strict Curve Scaling (disable loose...) has a better overall PSNR and has less "spikes" (higher min PSNR, lower max PSNR).
My personal (that means i could be wrong!) suggestion is to use Srict CS with sources with "constant" motion and Loose CS with sources with mixed motion scenes.
Decoder AR is b0rked.

@koepi: few more tests are coming. However the bitstream AR should be working (mpeg4modifier reads it correctly).

[Koepi]

Sharktooth:

I mean the dshow decoder. Does it display with "corrected" AR if there's a AR set in the bitstream? :)

Regards
Koepi

[Sharktooth]

Ehrr... ok... my fault... however the encoder works... :)

New Excel sheet with graph and Min/Avg/Max PSNR values (you can find them in the "Data" sheet below the PSNR values): http://ebola.gamersrevolt.it/koepi/xvid.xls

[Koepi]

Sorry to bump into my own thread again, but I just checked a sample - encoding anamorph with this build (thus writing the AR info into the bitstream) works like charme with the included dshow decoder.

So only forcing different AR during playback in dshow doesn't work, else everything seems quite alright :)

Regards
Koepi

[pogo stick]

Thank you, guys, for your hard work and testing.
I didn't try encoder yet.
I tried decoder. It's very nice to have brightness and deringing back. :)
But there is something wrong with auto-resizing. It's not working for mp4 files with B-frames. It's OK for avi, but not for mp4. Deinterlacer is not working for both of them. :(
I played XviD 1.0.x encodes in MPC and BSPlayer using 3ivx splitter.

[Koepi]

Deinterlacer only gets used if you a) checked the option and b) have the interlaced flag set in the bitstream(!).

I wonder why the mp4 AR isn't working, I guess it's something odd with the 3ivx splitter - it works fine with avi, ogm and mkv (thus really from the bitstream, not the conatiner AR).

Maybe you can switch something in the 3ivx splitter to disable it's AR features and thus let xvid take over (or: use the "auto(external first)" option in xvid dshow decoder to use the AR set by the splitter)?

EDIT: I have a 243kb sample which is made with bframes + 4:3-ar (352x576 resolution->768x576 corrected). Works like charme?!

Regards
Koepi

[bond]

afaik the 3ivx splitter passes the ar info by simply passing the vol to the decoder, which than leads to the resizing (i hope i remembered it right)
in .mp4 the vol is seperately stored from the raw bitstream (thats how the mpeg-4 spec wants it to be handled), so no vol attached to every i-frame aso, like its the case with .ogm, .avi and not native mpeg-4 (as all tools place mpeg-4 in mkv today) in .mkv...

btw koepi when i try to reach your page (koepi.org) i always get a "wrong page" message which than redirects me to koepi.org again, with the error message again aso...

[Koepi]

bond:

that's bad, then your provider (or router) doesn't properly refresh the DNS entries - it's been on the new servers for over 2 weeks now (I think already for close to 4 weeks).

You might want to call your ISPs customer service and ask them if they could refresh their DNS entries (at least for koepi.org). It should point to 198.87.84.234 (it's a vhost there at www.vogon.hr _ Daniel was so nice to host the site :) ).

On the issue with the non-working auto-AR with mp4 files, xvid doesn't check for a vol attahced before the bitstream anymore, we rely on the bitstream itself now. So doesn't the 3ivx splitter pass the vol in the bitstream itself? It should be perfectly valid to have the vol there in userdata?

Regards
KOepi

[pogo stick]

Originally posted by Koepi
Deinterlacer only gets used if you a) checked the option and b) have the interlaced flag set in the bitstream(!).
Yes, I understand, but it's not working. I even tried changing field order with MPEG4Modifier.
Originally posted by Koepi
I wonder why the mp4 AR isn't working, I guess it's something odd with the 3ivx splitter - it works fine with avi, ogm and mkv (thus really from the bitstream, not the conatiner AR).
Maybe you can switch something in the 3ivx splitter to disable it's AR features and thus let xvid take over (or: use the "auto(external first)" option in xvid dshow decoder to use the AR set by the splitter)?
Well, maybe it's because of 3ivx splitter. I don't know. There are no AR settings in splitter. It works for FFDShow.
I also tried "auto(external first)". All the same.
And I forgot to mention that auto-AR is not working for avi files without packed bitstream too. It's the same as with Minolta's hack.

[bond]

Originally posted by Koepi
So doesn't the 3ivx splitter pass the vol in the bitstream itself? It should be perfectly valid to have the vol there in userdata?i cant really answer this, i only know that in .mp4 the video data and the vol are seperately stored
i will point shitowax to this thread

[SeeMoreDigital]

Hi Koepi,

The new DSdec filter does not seem to be able correctly auto AR B-VOP encodes without packed bitstream :(


Cheers

[RadicalEd]

What's the current status of VBV? The last news on that was awhile ago and I was under the impression that work on it had stalled. Also, any information on the new rate control options? I recall some alternative rate control work done a few months back by an irregular (can't remember the name) -- is this the new strict cs?

[Sharktooth]

VBV seems to work. It's a checkbox in the 2nd pass options.
Rate control options (curve scaling) is also in the 2nd pass tab, you can choose a Loose or a Strict curve scaling.
Strict curve scaling tends to "average" the encoding quality (new default) while loose curve scaling is similar to the old (xvid 1.0x) curve scaling behaviour.

[minolta]

same here as pogo with deinterlace. checkmarked the option, but still no difference. perhaps changes in xvidcore.dll are not updated yet? also, can "deinterlace = on" be the default setting since most users don't configure the decoder?
thanks,
-minolta

[Leak]

Originally posted by Sharktooth
VBV seems to work. It's a checkbox in the 2nd pass options.

I've just noticed that while VBV seems to work the Level tab still says "XviD's ratecontrol will not respect these values" - removing that before a final release might be a good idea to prevent unneccessary confusion... :)

np: Delarosa & Asora - Saigon (Lily of the Valley)

[LoKi128]

As far as I understand, VBV is only needed (useful) when targeting standalones or otherwise bitrate limited machines, such as slow computers. For newer computers where bitrate dosnt matter, it should not be used.

Am I correct, or is VBV good for most encodes?

[RadicalEd]

Quality may be better without VBV because it can fluctuate more in a VBR environment, but then the stream isn't necessarily guaranteed to be decoded by a strictly compatible MPEG-4 profile@levelx decoder.

[BoNz1]

Hi, I have a request. I realize this is that 1.1 is beta so there isn't everything there that might be there if it was a release. But, I think it might be cool if in future versions we could have the DivX profiles selectable in the Profile at level. This would be cool. Also, the levels tab could be moved to the 2pass tab and things like the max frame size, buffer size could be editable. Just a suggestion. Thanks.

[LigH]

"VHQ for B frames too (mode decision only)" - that means: No matter if I chose VHQ 1..4 for P frames, B frames will use VHQ=1 (and 0 = off for both)?

Or in C style: "if (VHQp >= 1) VHQb = 1" (not "if (VHQp == 1) VHQb = 1")?

[sysKin]

Originally posted by LigH
"VHQ for B frames too (mode decision only)" - that means: No matter if I chose VHQ 1..4 for P frames, B frames will use VHQ=1 (and 0 = off for both)? No, setting for p-frames and setting for b-frames are independent. The wording is confusing, I know.

[LigH]

I was just confused if VHQ>1 is working with, and applied to B-frames too, although it sounded like "VHQ works in mode 1 only for B-frames, if used".

If I select VHQ = 4, mode 4 is applied to P-frames; but does it apply mode 1 or 4 to B-frames, if "VHQ for B frames" is enabled?

[yaz]

here are some evidence that all options work (i mean they do sg :-)
source : 1000 frames, mode=2pass, target bitrate = 3000, compression ~ 85%

set
I : no B-VOP
II : 1 B-VOP
III : 1 B-VOP + VHQ/B

cc
0 : 'inner' curve scaling
1 : 'loose'
2 : "strict vbv'
3 : 2 + 3 (! yes, in my version it's allowed)

results : max framesize / average br

I
0 -> 92340 / 3137
1 -> 92340 / 3209
2 -> 133477 / 2734
3 -> 133477 / 2769

II
0 -> 133477 / 3122
1 -> 133477 / 3138
2 -> 162148 / 2810
3 -> 133477 / 2748

III
0 -> 133477 / 3123
1 -> 133477 / 3139
2 -> 162148 / 2813
3 -> 133477 / 2745

as it's seen, the bitrate variation is fairly high (i haven't got time to try a bigger clip, but it'd worth)
notes :
1) the difference betw II & III is(!) significant as the psnr is always higher for III respectively
2) 'loose' scaling is pushing the quantizers downward. i mean, it uses lower quants than 'inner'. I-VOPs gets q=1 preferentially while 'strict' doesn't seem to like it (much less q=1)
3) the increase in max.framesize is always on an I-VOP but at different places for 'loose' and 'strict' (???)
4) the speed drop is about 20-25% when switching vhq/b on. (it'd be higher when more than 1 B-VOP used)
5) 'strict' changes the q distribution (not seen here) very much. i found even 2->8 shifts in q.

i found it a bit confusing (but maybe i was just too tired:-) a short explanation of the new opts would be quite beneficial. (maybe a devel or ...)

as regards dsf. yes, if b-frames are used wout packed bitstream then no aAR. that's pity.

the bests
y

[SeeMoreDigital]

Originally posted by yaz
...as regards dsf. yes, if b-frames are used wout packed bitstream then no aAR. that's pity. Agreed.

Just to make things 100% clear Koepi. The auto AR signalling still works in the bitstream because other software players such as Nero's ShowTime and VideoLAN can detect it. It's just the XviD DSdec filter that can't make it work.... yet!


Cheers

[Koepi]

The thing about the packed vs unpacked bitstream / bframe -auto-ar issue is a small one, I think we had a bugfix for that already but somehow I messed up (again).

I'll try to correct things with sysKin if he's finished with his tests and exams and wnats to spend some time doing bugfixing ;)

Also note that GomGom strongly votes against strict scaling, in his tests it always gives lower PSNR. (Which can be right, since the fluctuation of quality gets minimized we don't have near-black quant2 frames in an average quant5-encode anymore which raises PSNR quite a bit. It's all a matter of taste, and I think it should be possible to check the difference with a wider user base.). And remember, GomGom's mostly always right :) [I hope not in this case though as I like strict scaling very much ;) ]

Regards
Koepi

[Sharktooth]

Just leave the option there on the 2nd pass tab.
Let the users decide which scaling is better for their encodes.
That perfectly fits in the "xvid way"...

EDIT: Fixed typos

[CruNcher]

@Koepi

Foxers RC was allways under GomGoms in my tests that i did and i came to the same conclusion then GomGom especialy with harder 2pass settings instead of the standard 5/5/5 but you know that allready ;)

[Sharktooth]

Is it a big problem to have more than one standard curve scaling?

[Koepi]

No, it's about choice ;)

(May I repeat that PSNR isn't everything that counts? I can't stress that TOO_SMALL_LIMIT example which _always_ gave higher PSNR with visible degraded quality...)

I had a few words with GomGom and he just happens to not like strict scaling, so it's a not too serious issue (it's not that I totally crippled xvid now to say it with other words). Test for yourself and see what you like more. Though recommended because proven stable is of course the old loose scaling.

Regards
Koepi

[olnima]

First of all and I have to say it again and again:

THANKS TO ALL OF YOU WHO CODED, COMPILED, TESTED, RELEASED AND DID ANY OTHER KIND OF WORK ON THIS FANTASTIC PEACE OF SOFTWARE !!!!!!

One question about the 1.1-tree: Are there any improvements for
the "single-pass-directly-into-xvid-capture-users" like me?
(maybe with that kind of use I'm the only one who use xvid in a totally legal way :)

My settings are default except:
BVOPs off or 1 (because of playing back on DVD-player)
adaptive Quant is on (don't know if it makes sense, wanna save some bytes)
chroma optimizer enabled
MSP=5 ; VHQ=0 (because of my too slow CPU: AthlonXP 2000+)
Turbo-Mode is on when using B-VOPs
Min I/P/B-Quant is 2

Olnima

[Sharktooth]

Originally posted by Koepi
Test for yourself and see what you like more. Though recommended because proven stable is of course the old loose scaling.
More options, more fun... :)
However here are some results from SSIM tests on a High motion souce:
Xvid 1.02: Average SSIM= 65.57
Xvid 1.1 No B-WHQ/strict: Average SSIM= 65.41
Xvid 1.1 No B-WHQ/loose: Average SSIM= 65.63
Xvid 1.1 B-WHQ/strict: Average SSIM= 65.60
Xvid 1.1 B-WHQ/loose: Average SSIM= 65.84

As you can see the Loose Curve Scaling produce better SSIM results but the source is high motion...

PSNR:
Xvid 1.02: Average PSNR = 40,69843226 Overall PSNR = 39.8640
Xvid 1.1 No B-WHQ/strict: Average PSNR = 40,46406713 Overall PSNR = 39.9273
Xvid 1.1 No B-WHQ/loose: Average PSNR = 40,63747488 Overall PSNR = 39.8871
Xvid 1.1 B-WHQ/strict: Average PSNR = 40,51540425 Overall PSNR = 39.9988
Xvid 1.1 B-WHQ/loose: Average PSNR = 40,68424018 Overall PSNR = 39.9650

Also, Strict CS produces a LOWER AVERAGE PSNR but a HIGHER OVERALL PSNR indicating a more constant quality thru the whole encode.
I will try a low motion source to see if things will change.

[Koepi]

Thank you for verifying my point, as I wrote earlier, I think the quality is more constant with strict CC.

(Which then again is a matter of taste - quality fluctuation can look better. [That's why we added quantiser fluctuation to Nandub, it was giving back some "life" to the scenes - dunno if this really helps with XviD though]).

Keep the results coming! :)

Regards
Koepi

[Sagittaire]

18/10/04 ... ;)

Originally posted by Sagittaire
To make peace with Koepi:

I prefer a little worse Average PSNR (-0.1 dB or -0.2dB) if Min PSNR is very higher (+ 1dB). Average PSNR isn't really good tool for testing Rate Control quality ... ;)

Sharktooth's test confirm that: Average PSNR must be very better with CBR than VBR. For example sample with very long/very "low motion" part and very short/very "high motion" part done very better Average PSNR for CBR.

Overall PSNR or Frame PSNR graph are very better tools for testing RC quality. For example a Frame PSNR test had been able to show a weakness with the RC in H264 beta test from Ateme.

[Sharktooth]

More info:
I tried to playback my 1.1 encodes with the latest Nero Showtime.
Both Packed-bitstream and unpacked-bitstream were played back with the correct AR.

[SeeMoreDigital]

With regard to the DSdec filter. Is there any real need to have all those "aspect ratio" options.

Surly it would be less confusing if it operated fully automatically!


Cheers

[pogo stick]

How should VHQ for B-frames influence on file size? I made a few tests with Quantizer=2 and size of B-frames is always a little bigger with "VHQ for B-frames". But just increasing VHQ mode (without "VHQ for B-frames") making total size smaller. Is it right behavior? Or am I missing the whole point and there shouldn't be connections between file size and VHQ mode?

[Sharktooth]

@koepi: Can the other VHQ modes be implemented for b-frames? Have xvid devs done some tests yet and choosen VHQ-1 coz an higher VHQ is not woth the time spent for the encoding?

[SeeMoreDigital]

Sadly, I still can't make XviD 1B-VOP encodes work with my Xcard.

As I can't use B-VOP, can anybody confirm whether the SP encoding feature have been improved?


Cheers

[Manao]

pogo stick : VHQ is mainly about finding the best compromise between size and quality. Sometime, it's possible for VHQ to improve both quality and filesize, but most of the time, it has to choose either one or the other. To make a fair comparison, you then either have to compare two files which have the same filesize ( and in that case, the VHQ one will have a better quality ), or two files that have the same quality ( the VHQ one will be the smaller ).

Anyway, you'll find clips where VHQ for b-frames will reduce the filesize, and other where filesize will increase.

Sharktooth : yes it can be done. But are you ready to halve ( roughly ) the encoding speed of the codec ?

[Sharktooth]

Yes, if it's worth the time... obviously a 0.01db gain in PSNR will be useless. But if there's a more consistent advantage, well, why not do it?:D

[Manao]

I strongly doubt overall psnr will raise by more than 0.1 dB. But who knows... Anyway, since VHQ1 is implemented, VHQ > 1 can easily be added ( it's almost 'just' a matter of changing the CheckCandidate function by another one, already written ).

[Sharktooth]

a 0.1db is 10 times 0.01... :)
However b-frames are roughly 30-40% of a movie frames.
Rising b-frames quality/decreasing b-frames size keeping the same quality, will give a sensible boost to the overall movie quality.

[loni_blues]

Oh, where´s the file?
I feel like an idiot but at www.koepi.org I can find no updates since September 7. No single reference to the 1.1 build.
Am I nuts?
Regards,
loni_blues

[Sharktooth]

Go to xvid binaries section and find the latest UNSTABLE binary.

[loni_blues]

Thanks Sharktooth.
But no, nothing's there. I've all the same managed to download it by using www.koepi.org/XviD-1.1.-127-13102004.exe . I think it may have to do with the problem bond had: that's bad, then your provider (or router) doesn't properly refresh the DNS entries - it's been on the new servers for over 2 weeks now (I think already for close to 4 weeks). Anybody else with a similar problem?
Regards,

[malkion]

Thanks for the release Koepi. This version could really mess up my encodes, I luv that. :D

Anyways, I haven't experienced dropped frames in quite some time using xvid until I tried the 1.1 release.

Has anyone else had this happen?

[helix]

All the encodes i've done so far with 1.1 are superb, with no dropped frames at all. I did some anime and it seems to look slightly better with B-VHQ enabled. I haven't tried any encodes with the loose curve options but I want to see if that has any affect on quality.

[Koepi]

loni:

Hm, did you use the page out of your proxy or cache or something?

When heading to www.koepi.org or www.koepi.org/xvid.shtml , hold down the shift key and hit reload (and then release the shift key).
This will tell your browser/proxy to fetch the site fresh from the web.

I think this might solve your problem - that is a general problem with most of the internet users today, they don't know about proxies, caches and how they affect the browsing experience (no big problem usually, but as you see sometimes it's annoying [because of "false accusitions" ;) ]).
So the problem is totally different from bond's problem as you're just "seeing" an outdated webpage but access the correct server (you got the binary), bond has much more trouble, www.koepi.org points to the wrong server on the net for him, thus he can't even get the build.

I hope this helps

Regards
Koepi

[Boulder]

Originally posted by Koepi

I think this might solve your problem - that is a general problem with most of the internet users today, they don't know about proxies, caches and how they affect the browsing experience

Not to mention the fact that IE6 probably defaults to 'Automatically' regarding the situation when it should check for a new version of the cached page. Sometimes it just doesn't check for a new version for a very long time which then brings up the problem. That's why it should always be set at least to 'Every time you start Internet Explorer'.

To keep from going totally OT, thanks for the new binary! VHQ for B-frames didn't slow the encoding process as much as I figured it would:)

[yaz]

i can't get it. can anyone explain me how's this implementation work ? as i see from my table ticking different combinations of cc i got different results but ...
- what cc do i use if i don't tick any of the new options ?
- what cc do i use if i tick both 'strict' and 'loose' ? is it the case of 'loosely strict' or what ? :-)
- what conditions do these ccs try to maintain ? i went on testing and i saw that the new ccs always tampered with the bitrate curve even if a went down to 600kbps. as a simpleton i thought it's in the range where no real vbv maintenance's needed. such small frames wouldn't harm anything. or would they ?

sure, i misunderstand sg ... but what ?

thx
y