XviD-14052003-1.exe

[Koepi]

Ahoy mates,

since all of you can't wait until features are tested a little bit... here's a new build up. Don't complain about anything if something goes wrong! You have been warned! :)

Changelog:

XviD-14052003-1:
- Minor fixes & cleanups in the motion estimation routines.
- VHQ revised. Slower, but always increases PSNR.
- negative bframe threshold works.

Trellis based R-D quantisation
This feature on the debug-tab is at that location for a reason: it's more experimental. It only works with h.263 quantisation matrix.
It's supposed to set DCT-coefficients to 0 if the distortion introduced by that is lower than a weighted amount of bits saved (coding 0-coefficients is efficient.).
It'll introduce a slight drop in PSNR by it's nature, but the filesize gain overcompensates for that (in 2pass scenarios, that is.)

R-D based VHQ modes
This feature works on motion estimation. It means that the mode decision of a macroblock (intra, inter,...) is based on the weighted bits saved with using the distortion introduced by this decision.
It's slow, but effective (even in 1pass scenarios). PSNR always increases, file size shrinks.
It's a little less efficient in terms of file size saved compared to the "old" VHQ code at low quantisers (2), but saves more space and gives higher PSNR as the old code at higher quants (4 and up). This also means that VHQ is fixed for higher quantisers (was buggy before).

EDIT: in cvs, sysKin used a lambda value of 1.0 for weighting the bits saved in his R-D based VHQ code. I tweaked that value to 1.35 to achieve the abovementioned results - just that nobody can complain I don't spread the information about what's tweaked in my build vs. CVS ;)

I hope I explained the features satisfactory, hopefully sysKin will step in if something's inaccurate here. For not spreading all the useful info over 3 threads I'll close the other 2 threads where these features were started being "discussed".

Best regards
Koepi

[MoritzT]

Thx Koepi
keep 'em coming...........
:D

Cheers

eMpTy

[digitize]

Well done all xvid developers, including all the guys in the #xvid devel chan like cruncher, etc... The new vhq is very nice, vhq 4 is now the best, rather than 1, and although the compression gain isn't as nice the quality gain is better. Also with the lambda fix, compression and psnr are better at higher quants (if you're trying to compress a long movie onto 1 cd or something to that effect)(thanks to cruncher/koepi's work). Again job well done to all xvid developers!

[CruNcher]

Great work syskin & koepi :)
following Psnr tests where gathered till now :)


Devapi3
---------
New Vhq 4 Quant 2 Lambda 1.00 = 12,1 MB (12.728.320 bytes) PSNR = 46.3027
Old Vhq 4 Quant 2 Lambda 1.00 = 11,0 MB (11.609.600 bytes) PSNR = 45.9569
New Vhq 4 Quant 2 Lambda 1.35 = 11,8 MB (12.393.472 bytes) PSNR = 46.2397

New Vhq 4 Quant 4 Lambda 1.00 = 5,40 MB (5.671.424 bytes) PSNR = 43.8920
Old Vhq 4 Quant 4 Lambda 1.00 = 5,40 MB (5.670.912 bytes) PSNR = 43.8914
New Vhq 4 Quant 4 Lambda 1.35 = 5,11 MB (5.363.200 bytes) PSNR = 44.0136

Devapi4
----------
New Vhq 4 Quant 2 Lambda 1.00 = 12,4 MB (13.022.208 bytes) PSNR = 46.3102
New Vhq 4 Quant 2 Lambda 1.35 = 12,2 MB (12.804.608 bytes) PSNR = 46.2672

New Vhq 4 Quant 4 Lambda 1.00 = 4,27 MB (4.486.656 bytes) PSNR = 43.4467
New Vhq 4 Quant 4 Lambda 1.35 = 4,20 MB (4.405.760 bytes) PSNR = 43.3472


thx to md´ for his results too :)

EDITED by CruNcher: added Devapi4 New Vhq results

[Koepi]

@Cruncher:

Thanks a million for posting the results here as well. And thanks for testing the modified lambda.

I did nothing special here - kudos to you for mentioning the tweakable lambda (or was that sysKin himself? Ah, too lazy to check the log ;) ).

And of course a BIG thanks to sysKin for porting skal's/gruel's trellis code to cvs head, and for spending so much time to code fancy stuff like VHQ and r-d based VHQ :)

I hope this build is as good as the first tests promise.

Best regards
Koepi

[mf]

Originally posted by Koepi
I did nothing special here - kudos to you for mentioning the tweakable lambda (or was that sysKin himself? Ah, too lazy to check the log ;) ).

16:11 <@sysKin> Koepi: as you might remember, VHQ was computing number of bits and finding minimum. new vhq computes (lambda*bits + distortion) and minimizes that
16:44 <@sysKin_away> 'LAMBDA' #defined in motion_est.h
:D

[Koepi]

Thanks mate :)

[Selur]

Thanks for the new build!

[mf]

Originally posted by Koepi
Thanks mate :)
Lastlog to the rescue :).

[vinouz]

Damn cool work.... But it bugs...
[5 mn later...]
Lumi masking is (still ?) broken. [edit : on second pass]

So let's give it a try... !


Zou, Galinette

[Defiler]

I've read the relevant threads, but I haven't seen an answer, and of course it doesn't work as a search term.. What does "R-D" stand for?
Thanks for the new version.
Perhaps D stands for Distortion?

[Koepi]

Rate-Distortion

[HailMeBaby]

koepi, how do you know the word ahoy, is it german? coz it sounds czech

[mf]

Originally posted by HailMeBaby
koepi, how do you know the word ahoy, is it german? coz it sounds czech
It's a common greeting used at sea.

[TheXung]

Originally posted by HailMeBaby
koepi, how do you know the word ahoy, is it german? coz it sounds czech

ahoy is a nautical term and I believe it comes from old english.

[OBcecado]

Hi, thanks for the new build :)
I'd like to know if there is any more info about R-D quantization method, if not this link http://www.informatik.uni-freiburg.de/tr/complete_g.html (Report No.139, April 2000)


Thanks for listening :D

[MetaHiro]

It's taking me about 20 hours/pass with the new VHQ 4 to encode Lolita. This better make it better than the DVD itself ;)

[Sgt_Strider]

I wonder is it safe to use GMC now?

[cipher]

:)It's taking me about 20 hours/pass with the new VHQ 4 to encode Lolita. This better make it better than the DVD itself
Just for curiosity and a basis for comparison, what was it before? 16 or even 12 hours? :)

[cipher]

And yes, thanks for the new build!!!

[MetaHiro]

Originally posted by cipher
:)
Just for curiosity and a basis for comparison, what was it before? 16 or even 12 hours? :)

10 hours :P

[Koepi]

Ok, historical lesson:

"ahoy" is indeed the greeting used amongst sailors. "Ahoy" is historically czech, and it was used amongst christs there AFAIK. Not that I'm believing in anything (god, bible,... all such weird stuff).
Later on, the english adopted that as greet amongst sailors, too.

(I'm from the very high north of germany, a so called "Fischkopp". That's why I often use that. Even if i'm now living near the bavarian border and can literally smell the mediterrannian seas [Göttingen is in the south of nether saxony]) ;) .

@Sgt_Strider:
There's nothing in the changelog about GMC :P

Regards
Koepi

[Selur]

takes me something like 7 1/2 hrs for each pass on my dual athlon system (vhq4,chMo, chOpt, trellis, qpel) with 55-65 cpu usage,.. (it's the same speed if I need to ivtc, but then cpu usage is 100% ;))

Cu Selur

Ps.: Koepi: posted this also in the xvid.org forum, but are there any thoughts about optimising Xvid a bit for dual cpu systems? (there's still some cpu power left, if I don't ivtc :D)

[N_F]

@Koepi - I just got to ask. What's the deal with the "-1" on all your XviD releases?

BTW, is it just me who gets confused with the dates sometimes? I'd like to refer to the currect binary as 030514 (yymmdd), but I guess germans use ddmmyyyy. And some countries seems to use mmddyy which can be really confusing.

[mf]

Originally posted by N_F
@Koepi - I just got to ask. What's the deal with the "-1" on all your XviD releases?
There might be more releases on 1 day.

[N_F]

Originally posted by mf
There might be more releases on 1 day.
Ok. Has it ever happended or is it "just in case"?

[JagPanzer]

Yes it has. I have XviD-18112002-2 build on my disk. But I am sure that it happened more than just once. :)

[kilg0r3]

Originally posted by N_F
I'd like to refer to the currect binary as 030514 (yymmdd), but I guess germans use ddmmyyyy. I'd also opt for this file-name format. Although this is not a big issue.

[Blight]

Personally, I'd go with May14_2003 as to make it very clear, but then again, That's me.

[kilg0r3]

If we wanted to upset koepi we ought indeed continue this discussion about filenames. BTW, Blight, your alternative might cause artifacts in combination with Qpel and VHQ.:D

[iago]

Merhaba Koepi,

Thanks for the new build!

Btw, what about "XviD-thefourteenthofmaytwothousandthree-one" ? :D

regards,
iago

[vinouz]

it wouldn't burn with the entire name on an ISO archive of all the versions of Xvid codecs. (considering one doesn't want to use joliet for greater compatibility with old macs)

;o)

[Defiler]

Clearly,
XviD-TheFifteenthOfMay_InTheYearOfOurLordTwoThousandAndThree_LimitedEdition_FirstPressing.exe
is the most succinct and straightforward way to name these files.

[Selur]

for those who are interessted, seems like my problem is connected to:
VHQ 1: CPUusage 100%
VHQ 2: CPUusage 60-70%
VHQ 3: CPUusage 60-65%
VHQ 4: CPUusage 55-60%
(all other features don't change the CPUusage)

Cu Selur

Ps.: OS=Win2kpro

[CruNcher]

added Devapi4 Psnr results with New Vhq

[Koepi]

iago:

serefe dostum, it's a pleasure to serve new builds :)

Selur:

you mean while encoding on dual proc?

@all:

We had thos "name format" discussions more often. And just for the habbit of it, I won't change it, the ddmmyyyy-buildversion format is the most logical one :) (IMO of course. Please don't discuss that further as it simply doesn't gain PSNR or compression.)

Best regards
Koepi

[Defiler]

Selur: I've noticed similar behavior when encoding on SMP machines.
I'm doing one right now that's only using about 65%, with VHQ=4, Trellis, Chroma Enhancer, Chroma Motion, etc. However, the input Avisynth script is rather complicated, so take that value with a grain of salt. With VHQ=1, and simple Avisynth scripts, I usually see 100%, or close to it.

[Selur]

@koepi:
Yes, I get these values if I encode on my dual proc system.

using the following avs script:
LoadPlugin("C:\PROGRA~1\GORDIA~1\mpeg2dec3.dll")
mpeg2source("D:\mercury\m.d2v")
crop(0,76,718,424)
LanczosResize(672,272)

[Herculezz]

Beautiful New Build Love It :)

[HughMagoo]

for big rips (2CD, etc) is it recommended at this point to use H.263 in conjunction with trellis option, or just stick with MPEG or MPEG-Custom and wait for trellis to come into play later?

this would be in conjunction with b-frames,qpel,chroma opt, chroma motion, VHQ4, and either MPEG quant or something like HVS-Best..bitrate ~1300kbps.

thanks for any feedback.. build looks very sweet!

[symonjfox]

Hey guys, I discovered that if you stay on all options, the box has been updated ... it's more dettailed and is useful for newbies :o

PS: I was a newbie, but days on search on this forum ... made me happy :D

[bond]

huh it says that vhq 2-4 isnt recommended for low bitrates!?

[spyder]

@bond: The GUI needs revised. syskin's new VHQ will not hurt quality. It's safe for all bitrates supposedly. Unless of course a nasty bug has snuck in :)

Spyder

[R3g]

Sorry to ask it there, but I've been away for a while and couldn't find it anywhere : what about the compatibility between VHQ and other advance features (GMC, QPel and so on). And, by the way, is there still some feature combination that could lead to broken result for sure ?

EDIT : Oups, finally I found my answer about VHQ : it seems to remain uncompatible with GMC, but to work with other features.

[symonjfox]

Don't use VHQ+GMC

The rest hasn't changed.

[R3g]

Originally posted by symonjfox
Don't use VHQ+GMC

The rest hasn't changed.
Well, actually thet's my point : the rest hasn't changed, since when ?
The last time I came here, Lumimasking was broken, and, if I remember well, using GMC+QPel was at our own risk.

[mf]

Originally posted by R3g
Well, actually thet's my point : the rest hasn't changed, since when ?
The last time I came here, Lumimasking was broken, and, if I remember well, using GMC+QPel was at our own risk.
Don't use lumimasking or GMC, they're not effective. That's all you need to know :p.

[bond]

Originally posted by mf
Don't use lumimasking or GMC, they're not effective. That's all you need to know :p. or read the Newbie Settings (http://forum.doom9.org/showthread.php?s=&threadid=53136)

[Gazza]

I used the latest gknot to test this latest build from koepi. I used me, bframes (2/150/100/0), no qpel, mpeg, vhq=4, chroma optimiser everything else was at default.
The stream info file generated by DVD Decrypter showed delay for audio was zero. I generated an ogg file using the zero delay.

However, when muxing video & audio using oggmux (not Gknot)with zero delay, I noticed that the end result is not synched. I'm wondering if the 3 frames offset (-120ms) has crept in again. I am going to run another test later (when I have the chance) by manually altering the delay and see if this is it.

Anyone see this problem too?


[edit] koepi - video quality was very nice. Excellent job.

[edit] I adjusted the video/audio by 120ms and the result is good.

[Sharro]

Originally posted by Gazza

Anyone see this problem too?



Not in my last 10 encodes or so...
All the best,


Sharro