XviD-14022003-1

[Koepi]

Ahoy!

Good news everyone: sysKin played around again and hacked in some baaaad things ;)

Changelog:
- Activated experimental AC-HQ-PREDICTION by gruel.
- Added vfw support for sysKin's revised DCT ME (VHQ mode).
- Preliminary hack against dark blocks in bframes by sysKin.
Be warned: this is not a real fix! It might break things.

The warning is meant for real. It can seriously f*ck up your encode by adding new artefacts.

Still, please test!

Best regards
Koepi

[Mango Madness]

first reply!

Test: The Matrix
2 Pass encode
HQ Modulated
Mode 1 VHQ
Motion Search 6
Chroma, Lumi, and B-frame (3/150/100)
Using avisynth 2.5 beta with the latest virtualdubmod aiming for 650meg video plus a 150meg mp3 file (encoded with lame 3.94 alpha 11 vbr preset medium)

New toys are always appreciated.

[Koepi]

Please don't use modulated quantizers, they don't work correctly with bframes enabled. In that scenario i'd prefer H.263 quantizer matrix or even hvs_good_picture-matrix.
Mode 4 VHQ gives a better result in 2pass mode btw., the PSNR drop introduced is overcompensated by the filesize gain.

Regards
Koepi

[Sigmatador]

LOL a Saint Valentin's build :D

going to play with VHQ :cool:

thx ;)

[OUTPinged_]

the PSNR drop introduced is overcompensated by the filesize gain
err... what is "AC-HQ-PREDICTION"?


LOL a Saint Valentin's build
You noticed it too! Very, very suspicious. Gotta check the codec for hidden surprises..

[iago]

Yes, time to play with the new toy! Let's try to discover what the "AC-HQ-PREDICTION" is ;).

[Zarxrax]

Koepi, could you mind please explaining to us what those first 2 things do, the "experimental AC-HQ-PREDICTION" and the "vfw support for sysKin's revised DCT ME (VHQ mode)"?
Thanks :)

[Tommy Carrot]

AC-HQ prediction is linked to the keyframes (maybe all of the intra blocks?). Basicly the keyframes will be shorter about 1-2%.
(Don't hate me if i'm wrong. :D This is what i could understand from the dev-list)

[Koepi]

TommyCarrot's explanation is a bit simple, but mainly hits the point.

vfw-support for syskin's "dct ME" (old name, since it's all in CVS he changed it into bits ME as this is exactly what it does ;) ) is in CVS already and is another search refinement which reduces filesize by optimizing i.e. intra/inter macroblock decision based on the bits used and the higher VHQ modes (look into the VHQ manual thread) do another search to minimize the block size even more.

Regards
Koepi

[MaTTeR]

Originally posted by Koepi
Mode 4 VHQ gives a better result in 2pass mode btw., the PSNR drop introduced is overcompensated by the filesize gain. Has something changed? SysKin stated level 1 gives better quality and filesize overall. Maybe he was only speaking about the uManiac build?

[Koepi]

No. Mode1 improves 1pass quantizer/quality mode as well while modes 2-4 introduce a small drop in PSNR. In 2pass mode the filesize is so much less that with the additional space the PSNR after 2nd pass is higher than without those modes.

Regards
Koepi

[MaTTeR]

Cool..thx for explaining and the new build.

[Zarxrax]

This build just seems to crash vdubmod when I enable the VHQ. I tried VHQ level 1 and 4, and they both crashed, but it encodes fine when I disable the VHQ. I'm using h.263, motion search 6, and none of the extra options.

[Koepi]

Doesn't crash here and on other machines. What setup do you have? Did you push "load defaults" first?

Regards
Koepi

[csr2000]

I have the same crash,both on Koepi's and uManiacs' build.
OS is W2K Sp3
P4 2.4GA
Intel850
Samsung Rambus 800 512MB
WD 800JB * 2 @ Raid 0

Fisrt: load default, Crash
Second: Uninstall XviD,delet all reg key,reboot, reinstall XviD, the same Crash

I also heard someone suggest to disable sse2 in VDM,but always the same Crash.

Now I can only use XviD-02022003-1.exe

[Koepi]

Hum. P4 seems to be a freaky processor :-/

Anyhow... the good news:
Suxen_drol made chroma optimization an xvid-option via flags, so I added VfW support for it (see the debug-tab). Chroma optimization interpolates colour information in very bright or dark areas which helps stair effects on edges a bit.

XviD-15022003-1.exe (417kb)
Changelog:
- Fresh CVS checkout.
- Added vfw support for suxendrol's chroma optimizer.

Best regards
Koepi

[vinks]

both umaniac's and koepi's current build seems to have broken sse2 optimizations, both build crash on my p4 when sse2 is enabled. disabling it makes it all work :)

i guess its wait a few days till someone finds out whats broke and fixes it.

[csr2000]

please kindly tell how to disable SSE2?
in VDM,I disable SSE2 in options\perferance\cpu, but it seems not work

[Snakeisthestuff]

I have tested the 14022003 build on a short movie clip, which includes much movie-noise and where in later builds much dark blocks were produced by the use of b-frames.

my settings:
Motion search prec.: 6
quant type: h.263
VHQ mode: 4
use chroma motion, use gmc, use Qpel checked
max bframes: 2
b-frame quant ratio: 150
b-frame quant offset: 100
DX50 B-VOP comp. checked

results:
great! the dark blocks are gone, but like Koepi mentioned there are therefor some other artefacts :(

here a screenshot of some artefacts.

[Lefungus]

I also have a p4. If i want to use xvid, i must force cpu optimisations and disable sse2 instructions in xvid config box. The bug was here before (January) only if I got ref'divx lumimasking checked, but now it doesn't matter.

(Sorry for my poor english)

[zulu]

i just did a short clip and noticed artifacts in several scenes.
settings:
resoltion 512x272,
2 pass,
motion search 6,
quant type h.263,
vhq 1,
max I 250,
min i 1,
chroma motion,
qpel,
_no_ gmc,
b-frames 3,150,100
dx50 b-vop compatibility

screenshot attached.

[Sigmatador]

could you try the desativate each feature (one by one :cool: ) to find which one it is ? (only for this piece of video :D )

edit: what's your script and your xvid build ? ;)

[sysKin]

Originally posted by zulu
i just did a short clip and noticed artifacts in several scenes.
settings: Could you confirm that this is a b-frame? Use ffdshow's OSD. Well, I bet it is.

The warning at the very beginning, saying that this build can make such artifacts, was very real.

Radek

[csr2000]

Motion search prec.: 6
quant type: h.263
VHQ mode: 0
chroma motion on
gmc off
Qpel off
max bframes: 3
b-frame quant ratio: 100
b-frame quant offset: 200
DX50 B-VOP comp. checked

also can find the block like the picture show above

[OntzA]

It's because of bframes for sure. Yesterday I did some tests with and without bframes, and there always appear artifacts in bframes.
sysKin could you explain briefly why does VHQ fail with bframes?

[zulu]

Originally posted by sysKin
Could you confirm that this is a b-frame? Use ffdshow's OSD. Well, I bet it is.


you're right :D
ffdshow identifies this frame as "B QPEL".

[Koepi]

Erm. This isn't VHQ related. People, finally learn to read please.

That artefact derives from the hack which was meant to suppress dark blocks in bframes and VERY LOUD and CLEARLY got marked as HACK WHICH WILL PRODUCE ARTEFACTS.

_Nothing_ in conjunction with VHQ. Read me.

Regards
Koepi

[HarryM]

I found bug.

I use b-frames, q-pel. No WHQ (actually).
In b-frames are disordered blocks, mainly at high-speed, contrasted edges...
Koepi's build 02022003 is O.K., 14022003 and 15022003 are buggy.

I think, syskin's hotfix for darkblocks in b-frames is buggy possibly (see snap from xvid movie)...

[HarryM]

Ehhhh. Late. Zulu reported this bug too... :(

[angelyote]

I'm still getting the SSE2 crash in the 15022003 build. It's also in Umaniac's build and in the 14022003 build.

Same as with everyone else I can get rid of this by disabling SSE2 in the codec.

I know this has been reported already but I didn't see any response other than P4s being freaky processors.

@csr2000
In order to disable SSE2 you go into the debug panel of your xvid codec's advanced options and change 'automatically detect optimizations' to 'force optimizations' and uncheck SSE2.

Dave

[Koepi]

The SSE2 bug is known in the meantime. Unfortunately nobody has a real idea where it comes from.

Might be that noone of the developers owns a P4.

Koepi

[Lobuz]

It look's like this white pixels bug isn't in umaniac's 15.02.2003.0900 developer's version.

[BoNz1]

Alright, with all the talk about VHQ I have decided to do some testing myself. I am very impressed. The tests were done on the matrix reloaded trailer. The really cool thing about VHQ is that it is simple profile compatible which makes it very valuable. This is what I wanted to test, the reduction in bitrate using only simple profile features and VHQ. I used Koepi's 14022003-1 build. So here are my settings, everything I used was the same except for VHQ for each of the five times I did this, using res of 704x320:
Motion Search Precision 6
Quantization Type MPEG
Max I frame 300
Min I frame 1
DX50 B-VOP Compatibility
Now, here are the filesizes for the first pass the first one used VHQ 0, the second VHQ 1 and so on:
1. 33,019,904 bytes
2. 32,071,680 bytes
3. 31,381,504 bytes
4. 30,996,480 bytes
5. 30,767,104 bytes
Now, I really cannot see the difference between VHQ 0 and 4, I might use 4 for a 1-cd rip and 2 for a 2-cd rip since I get a substantial drop in fps. But as I was doing my testing I ran into something wierd. It has been reported here several times that there are problems with b-frames they get little blocks all over the place. Well, I didn't use b-frames or anything else for that matter and I still got this problem. When I did use b-frames, quarter pixel, chroma motion the problem was definitely worse though. I will attach some screen shots to show this. The first is from the source, the second is all advanced options disabled and VHQ 0, the third is with b-frames, quarter pixel, chroma motion, and VHQ 4. Hopefully, this will help to sort out this problem, I will test more if needed. I am also getting the problem with SSE2, disabling it everything works fine as reported earlier. Aghh, you can't attach much here. Oh well, I uploaded to a webpage so you can check it out here, http://www.geocities.com/bonzi5252/

[BoNz1]

Hmm, I was thinking about this again and I looked at my settings again. I could swear that I turned off b-frames but I think that I may not have. I am _very_ sorry if I caused any problems or misconceptions. If anything I guess it proves that 1/4 pixel seems to really amplify this problem as Keopi said before. Sorry again.

[Koepi]

XviD-17022003-1:
- Fresh CVS checkout.
- Better fix for dark blocks in bframes by sysKin.

It works perfectly for me now :)

Thanks for the code sysKin! :)

Best regards
Koepi

[sillKotscha]

thanks for your efforts but unfortunately it is not available...

but I can choose my meal (e.g. Halbes Grillhähnchen (3,4) an Paprikarahm, dazu Pommes frites und Broccoli, Dessert) by viewing the mensa menue of Studentenwerk Göttingen :D

[NuclearFusi0n]

We <3 you, Koepi.
You too, sysKin!

BTW, 404 on the new binary. You screwed the file name on the page link. ;)

EDIT by koepi:
DON'T spoof for downloading, I'll hit you with a foam bat(c)(r)[tm] ;)
/EDIT by koepi
EDIT BY FUSi0N: sorry man, i was too excited about the new build. like a child with a new toy! :D
question: since there are no ads on your homepage, how do you benefit from people visiting your page before downloading?




edit2: nevermind fixed, thanks koepi!!!

[Koepi]

OOpsi, sorry for the mess, i incidently hit "insert" button and thus deleted some characters which are vital ;)

Fixed!

And now guess what I'`m having for lunch today - i only have to walk 20 meters from here ;)

Best regards
Koepi

[NuclearFusi0n]

here goes testing

The Matrix
VHQ 4
QPEL
BFrames 3/150/100
MPEG quants
Chroma Motion
Rest on defaults


I'll update when it's done :)
currently have 9 hours remaining on pass 1, on my Athlon XP 2000+

[sillKotscha]

Originally posted by Koepi
And now guess what I'`m having for lunch today ;)

den Salatteller für eine Stamm2-Marke :D

/end of [german] chatting

[NuclearFusi0n]

Originally posted by sillKotscha
/end of [german] chatting
thankfully ;)


:p

[hayami]

Just tried the new build and the white artifact for bframe introduced in the last build is gone now. Thanks =D

[fraatz]

Hi Syskin & Koepi!
I just tested Koepis todays build and I still get dark blocks, the bright blocks are much more visible than with the original unmodified xvid code, too.
Anyway, thanks Syskin for looking into it!!!
fraatz

[Koepi]

You must be kidding, fraatz.

Please recheck your system, the build (17022003-1),...

I get no such problems anymore, really. I use my hardcore test which makes those blocks very easy visible, and it's more than fine now. The remaining "slightly darker blocks" which now pop up frmo time to time are dependant on the source.

Regards
Koepi

[Mgz]

Koepi, Would you like to put Xvid 0.9.1 offical on to your site. I really want to try it but I can't complie it :( (it said something about NMake,libxvidcore.mak,etc)


PLZ

[Koepi]

ß-9.1 isn't out yet. uManiac's scripts checkout the HEAD branch, not the appropiate 0.9.0 or 0.9.1 branch.

There's nothing stable yet, so i'll wait for that.

Regards
Koepi

[ookzDVD]

Just test Koepi's 17022003-1 build, with small clip.
VHQ=1 and ChromME enabled, B-Frame : 3/100/200.
The result is good as usual :)

[drebel]

Ok.Some more testing with Koepi's build (17022003-1) and syskin's VHQ
Results with 3 b frames 150/100,croma me,mpeg quant and 1st pass filesize :

-mode 0ff :9,08 mb
-mode 1 :8,78 mb
-mode 2 :8,63 mb
-mode 3 :8,56 mb
-mode 4 :8,46 mb
-mode 4+qpel:8,70 mb
-mode4+qpel+GMC:8,71 mb

and -mode 4 nob-frames,noqpel,nogmc : 12,1 mb

And some conclusions :

1. I couldnt find any of the "fast dark luma blocks" i mentioned the first time to syskin (nice job!!) ;)
2. I couldnt even see the "slightly darker blocks" that Koepi mentioned,even with 100 %zoom and full brightness.So,to my eyes,what i cannot see,doesn't exist :D
3. Each mode makes a filesize decrease with no jumps involved(mode1>mode2>mode3>mode4).So,is that a "hidden" bugfix? :confused:
4. The use of qpel increases filesize a bit (and decreases the effect of VHQ),but adds a lot to visual quality.So,it's really worth trying tha combination
5. Even though Syskin suggested not to use GMC with VHQ,the final avi was decodable with no probs at all.But the use of GMC didnt decrease the filesize any more.Visual quality was about the same
6. The use of b-frames helps a lot more to compressibility than VHQ alone.So, VHQ with NO b-frames is a waste of time and fps(my humble opinion ,sorry).But VHQ WITH b-brames adds a little something when it's really needed ...

Impressed by the innovations,
regards,
george

[Koepi]

VHQ doesn't work with GMC yet. You're lucky the encoding session didn't crash. :) But sysKin emphased that often enough for you to know this. ;)

Regards,
Koepi

[drebel]

Yea,i know...I was only playing around with the options . No crashes,however :)
And a question : does the stable build from umaniac (newest cvs ) has the same options as yours and working bugfree (as possible)?