VHQ manual

[sysKin]

[edit more than 2 years later]: this info is absolete. VHQ always improves quality, works with GMC and in fact also with b-frames, and it has no known bugs [/edit]

Hey.
I welcome every xvid alpha tester here.

As you might have noticed, the newest uManiac's build contains a new feature called VHQ.

VHQ stands or whatever you want, but Koepi said it's 'vastly hyped quality'. As far as I'm concerned it might mean 'very handsome qaligosaur'.

Important is what it does.
It does two things - first, it is able to compute the number of bits a macroblock would take in different scenarios and choose the scenario which shows the smallest number. Second, it can do some limited motion vector seach to minimize the bits for a given scenario. The 'scenarios' are macroblock modes - there can be one vector (we can look for it using this tool), four vectors (we can also look for them) or the macroblock can be intra (nothing to look for here).
As you can see (I hope?) it doesn't change quality at fixed quantizer - it reduces number of bits at fixed quantizer.

VHQ is only used in P-frames. In I-frames it doesn't have anything to do. It is possible in B-frames, but it's not there (yet? dunno, don't look at me).

It works with bframes, qpel, chroma, lumimasking etc.
It doesn't work with GMC yet. <-- until further notice. please remember that. I mean it.

There are several possible levels of additional search. Koepi and I decided to make four levels in the configuration - first is mode decision alone (without any search), next include additional search patters and are much slower.

OK, what have I forgotten?
Ah yes.
This is the first release and it's obviously buggy Current number of known bugs is: 0. Number of expected bugs is about 7649321 and a half.

If you feel like testing, you're welcome. Please share your comments and suggestions.
If you see a bug, do the following:
1. make sure GMC is disabled
2. check if it hasn't been reported before
3. report it here

And most importantly: Have fun testing.

Best regards,
Radek 'sysKin' Czyz

[drebel]

Reporting black(dark luma?)flashing macroblocks with version 0.2 1st pass(quant2),b-fr(3,150,100),cromaME,s.effort4.Filesize seems smaller compared to other buils with the same settings but quality isnt the same.But one should have a pretty sharp eye to tell the difference(at 25 fps)
Speed drop from 7-8 fps to 5 fps

[MaTTeR]

Testing a full 2-pass on two different machines. I'm anxious to see the results

Is there any room for speed improvement sometime in the future, in regards to VHQ? On both of the dual CPU rigs here, the CPU usage dropped from to about a fourth of what was being used previously. Just from what I read above I would have expected full CPU usage but then again I don't understand coding Thx for the new feature.

[Mango Madness]

Is it just me or is it nearly impossible to DL binaries from his site? Any mirrors available? I know this has been brought up before, if my memory serves me right. I shall commence testing once it finishes the infinitely (limit of x->0, oh wait, that's calculus crap) long download. Alpha tester, i like that title.

[angelyote]

Sadly this build (12022003 15:00) just crashes vdubmod and avs2avi no matter which options I selected. It crashed in 1-pass quant, 2-pass 1st pass.
*Did not crash during Null Speed Test but crashed when it finished.*

My most barebone attempt was:
loadplugin(mpeg2dec3.dll)
mpeg2source(mymovie.d2v)

and then ultra-high, h.263, b-frames off, all xvid options off (yes, gmc off) and VHQ at 0.

my system is a p4-2ghz, 512meg of memory and lots of gig of drive space free.

Dave

P.S. I will not be defeated. Gonna keep altering things until this works

[gamr]

nice results so far, about to try psnr tests to check that what im seeing isnt just me being optimistic but here's something to help those out there wondering if its useful

mode0 41084k
mode1 40048k
mode2 39622k
mode3 39378k
mode4 39298k

once i do the psnr ill report back again. nice compressability gain there. good work!

[seewen]

I just launched the 2-pass encode now, with VHQ mode set on "4 - Wide Search".
Other options : B-frames 3/150/100, ChromaME, MCP 6, DX50 B-VOP Compatibility = NO, Alt Curve = Medium ( default setting for the rest).

And the speed is quite the same compared with 2.2.2003 Koepi build ( same options, without VHQ ).

It's a very good surpise ( i was sure that the speed would be 1/10, or something like that )

AvScript

Mpeg2Source("F:\EOTS_D3_JAMES\XVID\Ennemi_Etat.d2v",cpu=6)
Crop(8,72,-8,-72)
Lumafilter(-2)
Undot()
SimpleResize(688,320)
Unfilter(10,10)
Limiter()

About 10-15 fps on an AMD ATHLON XP ( + Avisynth 2.5).


Bye, and thanks to all the dev !!


P.S.
Just made a short compressibility test, and it seems that "VHQ mode 4" make a gain of about 6-7% of compressibilty, maybe more.. But I'm not really sure of it ( I didn't remember exactly the result of the previous test )

[MaTTeR]

I'm seeing about a 4% compressibility change with mode 2 on a full encode two pass encode. Maybe it's just me but the visual quality looks great particularly the stable backgrounds such as walls. I'm noticing any flickering or moving blocks. Time to start another encode with mode 3.

BTW- The speed is more than acceptable here.

[ookzDVD]

I still have problem with uManiac's XviD.Alpha.12.02.2003.1500.exe build. The VirtualDubMod (31012003) suddently crash on the 2pass-1st pass. Avisynth 2.5 (dated 28/01/2003). MarcFD's MPEG2DEC3 (version 1.0).

Chroma, B-Frame(3/150/100), VHQ=4.

Even I've already clicked the "load default" button.

PS: Win2k SP3 on Intel P4 1.7Ghz.

[Bulletproof]

I just tried a Mode 1 encode w/ constant quantizer 2 and it gave me about 6% off the file. I was able to notice a degrade in quality, however it's not that bad and it's pretty acceptable. This should help alot in those long movies. All the new options have made XviD come a long way, although each option takes off a small amount of the size, when combined they offer great compression.

[vinks]

Originally posted by ookzDVD
I still have problem with uManiac's XviD.Alpha.12.02.2003.1500.exe build. The VirtualDubMod (31012003) suddently crash on the 2pass-1st pass. Avisynth 2.5 (dated 28/01/2003). MarcFD's MPEG2DEC3 (version 1.0).

Chroma, B-Frame(3/150/100), VHQ=4.

Even I've already clicked the "load default" button.

PS: Win2k SP3 on Intel P4 1.7Ghz.

i had a similar problem on my p4 as well, disabling sse2 sorts it out and all is well.

(running xp sp1 on a p4 2.4ghz (northwood) )

[ookzDVD]

@vinks,

Thank you for the reply, I just turn off the SSE2,
and everything is working well now
The Intel is office PC.

I just not try with my home PC yet, it's Athlon!

[Teegedeck]

4.5% filesize reduction with NO visible reduction of quality, here.

Encoded at quants 2 & 5, qpel and B-frames, with wide search range. Speed drop about 1/3. Now that's what I call an acceptable speed drop for the improvement, especially if I compare it to something called 'nth pass'...

Gain without pain. Thank you, sysKin! F-A-B-U-L-O-U-S.

[NuclearFusi0n]

Originally posted by Teegedeck
...I compare it to something called 'nth pass'...

ahahahahahah - it's so fun to chill in the DivX forums and laugh at them

[OUTPinged_]

to compute the number of bits a macroblock would take in different scenarios and choose the scenario which shows the smallest number. Second, it can do some limited motion vector seach to minimize the bits for a given scenario. The 'scenarios' are macroblock modes - there can be one vector (we can look for it using this tool), four vectors (we can also look for them) or the macroblock can be intra (nothing to look for here).

Doesn't that approach break compatibility with iso mpeg4(ie standalones)?

Did a quick check at q2 - it seems static scenes dont benefit alot from that - 3% for a clip with static pictures, 5% for himo-only clip at quant 2. I can see differences when i really look for them but i am doubt that would be noticable in dynamic.

(And IMO "VHQ" was a bad way to name it. Let me guess, 101% of idiots will enable it to 4th settings because they will think it means "very high quality".)

[Teegedeck]

Originally posted by OUTPinged_

Doesn't that approach break compatibility with iso mpeg4(ie standalones)?

No, as this is a more precise/economical mode-decision and search, nothing more. The resulting bitstream should be indistinguishable from an 'ordinary' encode.

(And IMO "VHQ" was a bad way to name it. Let me guess, 101% of idiots will enable it to 4th settings because they will think it means "very high quality".)

Something like 'very economic' would of course be more true to what it actually does. But on the other hand, if you go for a certain filesize it is a more 'HQ' way of reaching that filesize. Without VHQ, some higher quantizers would be used to reach that filesize, and this would be clearly noticeable.

[Didée]

Two questions:

First:
VHQ does some motion vector search, okay.
But what about the "normal" setting for ME precision? Is that related somehow, is there any influence, or does the setting for conventional ME not apply at all when VHQ is activated? Or works VHQ as a kind of pre-calculator for the normal ME?
Call me dumb, but this is not clear to me from syskin's explanation above ...

Second:
I noticed that when encoding a 3000frames-sequence with VHQ=4, final size ~ 30'000 kB, window's explorer showed me a filesize > 200'000 kB (!) during encoding.
How big may the file get during conversion? Could be of importance in low diskspace scenarios ...


Apart from that: hat off again, XviD team!!!

Regards

Didée


P.S.

"nth pass encoding" , huhuhuuu good joke, really ...

[OUTPinged_]

Ok, i wanted to make more tests. For some strange reason i had an impression that VHQ would work best on "hard to ME" material like shaky VHS captures with rotations, zooms and shakes. I picked up new year celebration DVD i had at hand (the cameraman was drunk so it was full of k3wl zooms and shakes :-) and checked with it.

VHQ=0: 6,50fps 5,552,128b
VHQ=1: 4.92fps 5,412,352b
VHQ=2: 3.58fps 5,251,584b
VHQ=3: 2.78fps 5,196,800b
VHQ=4: 2.20fps 5,067,776b

There. At quant2 filesize difference between 0 and 4 was 9.5%. Increasing quantizer to 5 it increased to 10.9% but when futher increasing quantizer to 10 it suddenly decreased to 4.7% - I dont know yet why did that happen.




About quality:

Comparing VHQ0 to VHQ4 PSNR (I know it's lame but anyway :-) was:
q2: 46.01-47.24 range
q5: 42.86-43.46 range
q10:39.90-41.87 range

Here are how the modes decrease PSNR (average for 10 frames):

Quant=2
VHQ0: 45.946239
VHQ1: 45.976748 - hehe. VHQ1 mode was better than VHQ0 here =)
VHQ2: 45.721406
VHQ3: 45.667344
VHQ4: 45.663510

Quant=5
VHQ0: 42.234175
VHQ1: 42.228793
VHQ2: 41.824608
VHQ3: 41.800283
VHQ4: 41.779857

Quant=10
VHQ0: 39.770136
VHQ1: 39.582636
VHQ2: 39.237580
VHQ3: 39.190024
VHQ4: 39.118455

At quant2 I couldnt see difference watching clip, and i could clearly point at differences only when zoomed it to 2x :/

At quant10 it was already crappy and VHQ4 didnt made it any worse :-) Actually i couldnt see it in motion, too.

My verdict: feature has great potential (Up to 10% bitrate savings!), but I should point everyone that some difference is present and it is possible that in some scenes there will be visible quality degradation. Some seroius testing is required here. It is slow, too. I dont see how people manage to get away with only 30% slowdown, since i am getting 6->2 fps drop here :-( I hope that futher optimisation is possible for VHQ (expecially mode4).

[symonjfox]

Originally posted by OUTPinged_
(And IMO "VHQ" was a bad way to name it. Let me guess, 101% of idiots will enable it to 4th settings because they will think it means "very high quality".)

So 4th setting doesn't always give the best quality? I didn't have any time to test it, yet.

Isn't it possible to add an option called 5 - AUTOMATIC where it performs all 4 possible alghoritms and then just uses the one that gave the best quality in that particular scene? It will slow down the encoding (I know), but it should improve the quality.

Another idea is doing something where, performing the first pass of a 2 pass encoding, it saves a VHQ.STAT where there are the results. When performing the 2nd pass, it will speed up because the Xvid already knows the best way to perform the VHQ algorithm, so ...

I studied a little bit of programming, but I'm very very far from being a programmer ... just the idea should be nice (don't you think)?

[sysKin]

We've been doing some tests as well. The results are similar:

Level 1 gives both better filesize and better quality (psnr).

All next levels decrese filesize more, but also decrese quality. This is all true.

At this moment I'd say that level 1 is a winner. But, as I said, it's work in progress

Radek