Custom Quant Matrices + VHQ and Other Potential Problems

[kilg0r3]

Hi!

Since I have been playing around a little with custom matrices, I am interested in the problems that can arise by using them, e.g. mp4 incompatibility(?), incompatibility with ffdshow (there was something ..) etc.

My first specific question regarding compatibility issues concerns xvid's vhq feature. I am not really familiar how it works. But from what I have read so far it seems to be a potential trouble maker with custom quantization. Does anybody know: Is vhq compatible with custom matrices?

[sysKin]

Originally posted by kilg0r3
Since I have been playing around a little with custom matrices, I am interested in the problems that can arise by using them, e.g. mp4 incompatibility(?), incompatibility with ffdshow (there was something ..) etc.
MPEG matrix (either default or not default) is a part of mpeg-4 advanced simple profile. It should be decoded by every decoder which decodes this profile.
Ffdshow seems to work great. I'm not aware of any problems of this kind.
My first specific question regarding compatibility issues concerns xvid's vhq feature. I am not really familiar how it works. But from what I have read so far it seems to be a potential trouble maker with custom quantization. Does anybody know: Is vhq compatible with custom matrices? Yes, there shouldn't be any problems with VHQ. VHQ uses standard quantization functions, exectly the ones used for coding later - if they work with mpeg-custom, they work with mpeg-custom ;)
For more about VHQ do a search for "vhq manual"

Radek

[trbarry]

I've just recently heard of problems using custom matrices with ffdshow.

I wonder if one of the problems is the max value that may be placed in a matrix. When I was playing with making the Soft_Matrix, etc., I asked what the maximum number could be but didn't get any definitive answer. So I plugged in (IIRC) 255 to some slots to make very high quant values. I guess it's possible this is not valid.

Can anybody clarify for sure what the max value should be for MPEG-4 quant matrices?

- Tom

[Selur]

Hmm,.. I just did a test with:
chroma optimizer+ chroma motion+ vhq(4)+kilg0r's "Mpeg-in half" matrix
@1pass quantizer 2

And guess what?
The size didn't change compared to the normal mpeg matrix.

So could it be that if one uses VHQ custom matrices don't do anything ?

Cu Selur

[kilg0r3]

@selur

that matrix is crap. DON'T use it for anything.

[MemTronic]

1. Xvid in it's current state is PAR EXCELLENCE! ...really impressive quality! ...hat off devels!!!

2. With some matrices you WILL have trouble with actual ffdshow 03-01-2003
__ The only matrices I am able to use (when played back later over ffdshow)
__ are = the 3 hvs_xxx matrices from RefDivX (btw. EXCELLENT work RefDivX!)
__ other tend to produce black macroblocks during playback!
__ ...it seems (for me) to be related with TOO LOW set numbers (no matter if inter or intra)
__ but really have not experimented that much with it lately!
__ (hvs_better_matrix fits my needs perfectly ...so I was just playing around a bit with the others)

3. VHQ causes NO troubles with any matrix which works without VHQ!
__ Extremely tested so far.


so long...

MemTronic

[Selur]

Hmm,.. tried the hvs_best_picture matrix, but still no size difference
(chroma optimizer+ chroma motion+ vhq(4)@1pass quantizer 2 )with uManiacs current build.

Cu Selur

[Bulletproof]

VHQ + Custom matrix is working just fine for me in Nic's build, try using the high compression matrix I did that should show a significant difference in filesize.

[Selur]

Doh,.. found the bug,.. after loading the defaults again and doing a test everything works like it should ;)
Sorry :(

Cu Selur

[JasonFly]

I have just made an encode with cgi matrix for Final Fantasy(I 'mp not sure it is worth for this sort of "realistic" cgi) with vhq and it looks very good.
I don't use ffdshow and all was well with Nic's decoder

[Didée]

Originally posted by Selur
The size didn't change compared to the normal mpeg matrix.
...
Hmm,.. tried the hvs_best_picture matrix, but still no size difference
...
Doh,.. found the bug,.. after loading the defaults again and doing a test everything works like it should
Cu Selur
It may happen from time to time, that the usage of custom matrix gets "locked" within the codec settings.
E.g. after playing with mpeg-custom, even if you switch back to "normal" mpeg quantisation, still the last loaded custom matrix gets used.
I have reported this several times now, but everytime there was either no response at all, or something like "no, that can't be."
The tricky part is, that behaviour is not easy to reproduce. It just happens from time to time.
These days, I always have a link on my desktop to the latest binaries installer ... ;)

Didée

[TNM]

Originally posted by JasonFly
I have just made an encode with cgi matrix for Final Fantasy(I 'mp not sure it is worth for this sort of "realistic" cgi) with vhq and it looks very good.


I'm very interested in this cgi matrix. Could you please tell me where to get? I did a search here with key words "cgi matrix" but couldn't find it :(

[Didée]

TNM,

search for "CG animation matrix", it was somewhere in a custom-quant-thread ...

[TNM]

Originally posted by Didée
TNM,

search for "CG animation matrix", it was somewhere in a custom-quant-thread ...
Thanks a lot Didee, I got it already. Gonna try it with some PS2 game movies to see how :D

[drebel]

Tests have been made with XviD settings :
(b-frames,chr.motion,chr.optimizer,m=6,VHQ) +Qpel



MATRIX ______COMPRESSIBILITY(%) ______FILESIZE(1 PASS)-MB


Andreas78 _________62,1 _____62,5_________________23,1 _____23,0
HVS-best matrix _____79,4 _____79,6________________18,1______18,0
Mpeg _____________81,5______82,9_________________17,6_____17,3
H263______________94,0_____100,0_________________15,3_____14,4



Some concusions( IMHO)

Mpeg-custom bug STILL exists.When a custom matrix is used for the first time and then mpeg (simple) is used again ,the custom is prefered(no respect for the "MPEG" tab).Even if H263 or other is used in between
H263 is the smoothest of all(just a confirmation)and gives a "flat" impression of the picture
Hvs-best gave a comp test close to mpeg ,but i prefer Mpeg when Qpel is not used(gives me less artifacts)
Qpel enhances detail,but also noise.It gave me a comp boost of ~0,5% (except for H263: 5% is a HUGE difference .Doublechecked though...)
The combination that came closer to avs was (in that order)


andreas78+qpel
HVS-best+qpel
H263+qpel

Mpeg+qpel was the worst combination by far.Mpeg4 artifacts allover.


So,about Qpel: in a NON compressible movie ,i really like H263+qpel.If it compresses well(or i have a lot of bitrate-space to spend),i prefer Andreas78+qpel or HVS-best +qpel.Without Qpel, mpeg is still the best for me.

PS:I dindnt use PSNR,only my eyes ;)
Optical conclusions have been conducted with quant2 from 1st pass,VirtualDubMod,1" monitor,X2 zoom,lanczosresize,Unfilter(5,-5)

regards,
george

[crusty]

Just wondering: why did you use that particular unfilter setting ?

[drebel]

@crusty

Artificial Edge Enhancement during mastering procedure is much more irritating to the human eye in the vertical axis (stairstepping effect),thus trying to reduce the effect there and increase it in the vertical axis (for sharpening) seems a nice idea.And ,of course, it takes good care of the "blocks in dark areas" issue.

[mikeson]

@drebel:
Mpeg-custom bug STILL exists.When a custom matrix is used for the first time and then mpeg (simple) is used again ,the custom is prefered(no respect for the "MPEG" tab).Even if H263 or other is used in between
And what if you 'Load Defaults...' between encodes?

[kilg0r3]

you could also export the default mpeg matrix and load it after you used your custom matrices. btw, does anybody know what the h.263 matrix looks like?

[drebel]

@ mikeson
From what i can remember,the "load defaults" hit was the first thing i did and was not enough.Only a re-installation of XviD solved the prob

[kxy]

drebel,

I am not doubting you. But I want to see it with my own eyes. Can you post some screen shots? I didn't know Mpeg+qpel produces worser result than h.263+qpel.

[mikeson]

I didn't know Mpeg+qpel produces worser result than h.263+qpel.
Didn't know either, but I can confirm that. Why is that?

[JasonFly]

I have made a test encode of the bourne identity with HVS best with and without qpel and the result wasn't what i waited for.
There were many artifacts even when i looked the (2pass: 1pass) result in q2.
The movie had much noise and it's one of the ugliest rip i ever did.
There were also other artifacts. But i cannot manage to see if they were due to Xvid uinstable or avisynth 2.51 or some avisynth filter.

[mikeson]

@JasonFly:

AviSynth doesn't introduce artifacts. What AviSynth filters did you use? What XviD settings are you encoding with?

[drebel]

You 'll have your screenshots by tommorow(i have to find a nice server to upload those .png's ,first).As i said ,qpel enhances noise as well.That means one should have to de-noise pretty well before touching that button.Playback performed with xvid.dll (no pp on encoder options)

regards,
george

[drebel]

The screenshots i promised

http://drebel.topcities.com/H263.png
http://drebel.topcities.com/H263+qpel.png
http://drebel.topcities.com/mpeg.png
http://drebel.topcities.com/MPEG+QPEL.png
http://drebel.topcities.com/HVS-best.png
http://drebel.topcities.com/HVS-best+qpel.png
http://drebel.topcities.com/Andreas78.png
http://drebel.topcities.com/Andreas78+qpel.png
http://drebel.topcities.com/Source.png


Conclusions are yours,now...;) :p
Tests with VHQ=1(with latest binaries commited in the cvs by syskin),3/126/100 b-frames and convolution3d(medium settings),Undot(),Lumafilter(),LanczosResize),Unfilter(5,-5)
Those are KFrames(sorry about the filesize:~500 kb each) :o
Geforce2mx 32 mb(modified for RFI filtering,see here (http://www.geocities.com/porotuner/imagequality.html) ,detonators Omega KX 1.4230,dx9,monitor AOC KRL+17"

Final encode was made with Andreas78+Qpel+bframes,and deen(a3d) coz convolution was too "gentle" with the noise as you can see.
Mirror shortly if possible

regards,
george

[JasonFly]

I usually only use convolution3d and this particular time i have also used unfilter.
The xvid setting are.VHQ1,Qpel, CME, and two encodes with Custom HVS best, and classic mpeg matrix.The results were not the same but the problems ihave talked about were in both encodes.

But i'm wondering if it's not an xvidproblem because this look like a bframe problem.I havn't used them in this particular encode but i'm wondering if the -1 setting is 100% effective.
I also had a problem of color which somtiomes turn to pînk(just a little).Where could this problem come from?Avisynth?Xvid ? Or my filters.
I'll try to post screenshot later

PS:
I used Avisynth 2.51beta.It is still a beta, so there could be some problems but I suppose we(users of avisynth) would have seen this sort of bug before.

[TCD02]

Hi,



I also had a problem of color which somtiomes turn to pînk(just a little).Where could this problem come from?Avisynth?Xvid ? Or my filters.

Make the same encode without filters...
Convolution necessary??
With which app do you do compression, VDub Mod??



But i'm wondering if it's not an xvidproblem because this look like a bframe problem.I havn't used them in this particular encode but i'm wondering if the -1 setting is 100% effective.


VHQ only works for P-Frames, which means it doesn't affect any B-Frames, so don't use it when encoding w b-frames (you havent used in this encode, i know). It could always result in various problems. VHW for b-Frames is under Developement, i think.

And...
GMC and VHQ counteract, so i would not use them together.

I'm not sure, if i'm up with the latest developements, so please teach me if i'm not.

Greets TCD

[Koepi]

Originally posted by TCD02
VHQ only works for P-Frames, which means it doesn't affect any B-Frames, so don't use it when encoding w b-frames (you havent used in this encode, i know).

*caugh* I use it in mostly every encode, including bframes.
Just because it's not developed for bframes it doesn't mean you shouldn't use it when you use bframes.

In the contrary, I don't expect it to be efficient on bframes, there won't be much size gain - thanks to the skip-mode with a TOO_SMALL_LIMIT which sysKin implemented for bframes.

VHQ is just a more precise search - so hell, if you have the time, USE it - if you use bframes, use VHQ anyways. No contradiction in that.

Koepi

[sysKin]

Originally posted by TCD02
[B]VHQ only works for P-Frames, which means it doesn't affect any B-Frames, so don't use it when encoding w b-frames (you havent used in this encode, i know).
OK I'm in a bad mood today, so I am unable to make fun of it. Let me just tell you NOT TO LISTEN, it's completely untrue, absolutely untrue, and I can prove why it's the contrary - VHQ works even better with b-frames than without them.
VHW for b-Frames is under Developement, i think. Not really... but I can do that... why not... Mostly just copy&paste.

Radek

[TCD02]

Hi,
thanks a lot, but don't be too nice, you could hurt yourself.
I'm always looking forward to learn.

I took this suggestion from the German FAQ (Wissenswertes rund um DivX).
And i posted a question in the german forum, too.

So theese FAQ are just a little bit outdated (04.25.2003):)), or just false, in a little detail!!
[URL=http://forum.doom9.de/showthread.php?s=&threadid=29[/URL]

"A contradiction between VHQ and GMC is mentioned there.
And a suggestion, b-Frames shouldn't be used with VHQ because it could lead into problems"
Da VHQ momentan nur für P-frames arbeitet scheint es Probleme mit VHQ+B-Frames zu geben
Da GMC und VHQ sich entgegenwirken sollte man sie nicht gleichzeitig aktiviert haben. Je höher die VHQ Stufe ist die man nimmt desto extremer wird der Encodingvorgang ausgebremst.


Please have a look at theese if you can find the time, if at least one of is/knows german.:)

Koepi: Just because it's not developed for bframes it doesn't mean you shouldn't use it when you use bframes.
Sure. I knew.


In the contrary, I don't expect it to be efficient on bframes, there won't be much size gain - thanks to the skip-mode with a TOO_SMALL_LIMIT which sysKin implemented for bframes.

Am i getting this right.
So the VHQ just affects p-Frames, but using b-Frames, which aren't affected by VHQ, won't lead into problems
Or does it affect B-Frames, and if yes, won't lead into problems, too?


Koepi: Is there probably someone, if not you, that could update your help file/options explained file.
Tool tips seem to be enough, though.

Thank you for your answers.
And especially thank you for your past and ongoing work on the codec.

Greets,
TCD

[TCD02]

By the way,
i couldn't neither find a link nor a picture, which shows the DCT coeff. of the "custom matrices" mentioned in this thread.

Can anybody help out?

Greets,
TCD

[sysKin]

Da VHQ momentan nur für P-frames arbeitet scheint es Probleme mit VHQ+B-Frames zu gebenDoes it say so in german forum? That's not true.
Let me explain: VHQ works on P-frames, which simply means it doesn't work on B-frames or I-frames. Just like that. Your video will have all these types of frames, and VHQ will be in effect when coding P-frames... This is where VHQ should work (at least in the first place) because P-frames will always take almost entire bitrate - B- will be small anyway, while I- will be rare.
Originally posted by TCD02
So the VHQ just affects p-Frames, but using b-Frames, which aren't affected by VHQ, won't lead into problems
Yes, this is correct.

Regards,
Radek

[Selur]

"Da VHQ momentan ..."
Think I wrote it,.. ;)

And I'll update it somewhere during this week,..
(better I'll delete the post and replace it with a total new (updated) document)

Thanks for syskin to clarifly that:
There's no problem with vhq and bframes,.. :)
But when I wrote the stuff there clearly was,.. picture really got a lot of macroblocks when I tested the combination (bitrate>900).

Cu Selur

[kilg0r3]

Anyone noticed that the good-picture matrix kills quite a lot of detail even at q2/3(b-frame)? The faces in 'Die Sass Brüder' were all smoothed. And the picture looked somewhat blurry. Well, I guess that is the price for a 107min movie having a first pass size(!) of 590MB :D with 2 bframes at 150/100/100.