Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
12th February 2003, 00:58 | #1 | Link |
Registered User
Join Date: Oct 2001
Location: Gainesville FL USA
Posts: 2,092
|
Free filtering using custom quants?
When you have a noisy source or are going to encode at a very low bit rate sometimes it's better to filter the heck out of it first, removing high frequency components that will cause artifacts and be hard to compress.
It's come up a couple times here before that it might be possible to just use Xvid custom quant matrices to sort of do this for free, with no run time CPU cost. I asked about it a couple times last summer but it seemed custom quants were busted then. But since then they've apparently been fixed and this morning I played with it for awhile. For an extreme case I made quant matrices that had the value 99 in most all but the top left few positions (see attached file SoftMatrix.zip). Then I made a 1280x720p 1 pass CBR of the recent HDTV Daredevil trailer at only 2.5 mbps, usually too low for most HDTV. It actually came out pretty good, better than I had expected. Can anyone tell me what max number I'm allowed to use in Xvid custom quant matrices? I just filled in 99 and it seemed to work but I don't see it documented anywhere. I want a number that just forces some DCT components to zero as much as possible. - Tom Last edited by trbarry; 12th February 2003 at 01:01. |
12th February 2003, 03:00 | #2 | Link |
Flying Attack Donkey
Join Date: Mar 2002
Posts: 159
|
Hmmm.... Don't remember exactly, but the XviD sources have the quant matrices stored as 1 byte, limiting the max quant to 255. This should probably make most values equal to zero, since most things which might be bigger than 510 are in the real upper-left of the DCT-domain array; most other things don't transform back to valid spatial-domain arrays.
__________________
"Bork Bork Bork" |
12th February 2003, 06:44 | #3 | Link |
The frumious Bandersnatch
Join Date: Sep 2002
Posts: 78
|
I'm sorry this doesn't really help Tom with his search for maximum values for quantums. I hope it adds to the useful input this thread will produce.
I'd seen some requests by people for information about how custom matrices work but I didn't see any information. Though they seem powerful and sound interesting I think they remain a mystery to a large percentage of the people here. Here is a link I found that I thought really gave me a clear picture of what the arrays mean. http://www.ece.purdue.edu/~ace/jpeg-tut/jpegtut1.html Dave EDIT: This post is a complete re-write. I thought the information would be handy as a link instead of a quote from one of its pages. My apologies to anyone who responded in it's original shorter form. Last edited by angelyote; 12th February 2003 at 07:03. |
12th February 2003, 12:10 | #4 | Link |
Registered User
Join Date: Oct 2001
Posts: 80
|
I'm not sure this would work, but maybe I'm totally wrong...I hope my poor english is enough to explain what I want to say...
I read somewhere (probably in this forum) that for effect of rounding (to not propagate error) the values stored would be 0 1 0 1 0 1, ecc. and not always 0. I've searched the post I'm talking about with no success. Maybe some of the developer should answer. Last edited by CavalloPazzo; 12th February 2003 at 12:13. |
14th February 2003, 17:16 | #6 | Link |
Registered User
Join Date: Feb 2002
Posts: 1,195
|
@trbarry
could i feel free to add your matrix in my tool ? (i have added support of 9 custom matrix based on the file made by reference divx).
__________________
AutoDub v1.8 : Divx3/4/5 & Xvid Video codec and .OGG/.MP3/.AC3/.WMA audio codec. AutoRV10 v1.0 : Use RealVideo 10 Codec and support 2 Audio Streams and Subtitles. |
14th February 2003, 17:37 | #7 | Link | ||
Registered User
Join Date: Oct 2001
Location: Gainesville FL USA
Posts: 2,092
|
Quote:
Thanks for the ref. I was following that thread at the time but Xvid then still had some bugs relating to custom matrices. But I'm also diverging from that viewpoint in another way. Most of those custom filters are trying to find an optimum blend of how to smoothly increase the quant factor from low to high frequency components, and this is good. But there is no free lunch. I'm instead proposing to decide up front how much high frequency detail we are willing to sacrifice (or think is noise and artifacts anyway) and not try to keep it at all. In the worst case this will lose detail on easy to compress still scence but avoids the artifacts that you get with quantizers over 20 or so. I do this just by setting all the lower diagonals to 255. This will hopefully drive all/most of the values for those high frequences completely to zero in the DCT result. I'd even set them to 9999 if Xvid allowed it, but it doesn't seem to matter. It would be neat if this could be used with modulated quants somehow. Koepi - Is this possible? Quote:
Dark-Cracker - Yes, of course. But thanks for asking. However I've also attached a slightly less extreme case (Filtered_Cutoff_Matrix) that I think works much better. I'm probably going to start using this one for many of my future encodes. So you might want to try that one first. It's my best so far. - Tom |
||
14th February 2003, 18:23 | #8 | Link |
retired
Join Date: Jun 2002
Location: hollywood
Posts: 1,013
|
@Tom
I'm waiting for the attachment to get approved, I'm really anxious to test the new matrix of yours. The previous one gave me very visible edge artifacts in a test encode on Natural Born Killers using quant2[edited] . (Btw, for some extreme cases, that is for some "mission impossible" encodes when aimed for 1CD , I have been getting really good results with DctFilter using some aggressive settings together with a combo of FluxSmooth/BilinearResize/cpu=6 (with Marc's mpeg2dec3.dll) in order to get rid of edge artifacts. Of course the result is really smooth, but it's at least free from blockiness and other artifacts, and for the time being this method seems to be the only solution for me when it comes to some movies that are otherwise "virtually impossible" to put on 1CD, such as Natural Born Killers .) thanks for all your effort, best regards, iago edit - but I must admit that I used b-frames as well . Last edited by iago; 14th February 2003 at 18:38. |
14th February 2003, 18:36 | #9 | Link |
Registered User
Join Date: Oct 2001
Location: Gainesville FL USA
Posts: 2,092
|
"The previous one gave me very visible edge artifacts in a test encode on Natural Born Killers using quant2/h263."
I'm not sure I understand here. Did you have H263 selected for the quant type, or MPEG-custom? I though only the latter would actually use whatever you loaded for a custom matrix. - Tom |
14th February 2003, 18:38 | #10 | Link |
retired
Join Date: Jun 2002
Location: hollywood
Posts: 1,013
|
@Tom,
Sorry, of course you are right, a terrible typo. Sure I used mpeg-custom loading your soft matrix with Nic's latest build. (I have corrected that mistake in the post above.) Last edited by iago; 14th February 2003 at 18:42. |
14th February 2003, 18:51 | #11 | Link |
Registered User
Join Date: Feb 2002
Posts: 1,195
|
hi,
your matrix seems to be good for low bitrate , i have tested on "swordfish" i have lost some details but the final filesize was lower than h263 encode. 1pass, 10 Mo wanted, 640*272 , bitrate 650 kbps. h263 => 9.85 Mo softmatrix => 9.76 Mo here some screenshot (screen vdub X 2) http://www.eclipsedvd.firstream.net/...rix/1-h263.jpg http://www.eclipsedvd.firstream.net/...softMatrix.jpg http://www.eclipsedvd.firstream.net/...rix/3-h263.jpg http://www.eclipsedvd.firstream.net/...softmatrix.jpg u lose some details i think this matrix will be good for hight resolution (i think less block) or for anime. i wait the moderator accept to test the next matrix, less strong. [EDIT] no b-frame used only chromaM and search : 6, MinMax i : 2/4 MinMax p : 2/12, MinMax kf inter. 10/300 frm. [/EDIT] Bye.
__________________
AutoDub v1.8 : Divx3/4/5 & Xvid Video codec and .OGG/.MP3/.AC3/.WMA audio codec. AutoRV10 v1.0 : Use RealVideo 10 Codec and support 2 Audio Streams and Subtitles. Last edited by Dark-Cracker; 14th February 2003 at 18:59. |
14th February 2003, 19:03 | #12 | Link | |
Registered User
Join Date: Oct 2002
Location: Portugal
Posts: 1,182
|
Quote:
Unfilter(-5,-5)/FluxSmooth(5,5)/Unfilter(-5,-5)/cpu=0 and B-F (3/150/100) also too smooth, but no MB visible. Bilu Last edited by bilu; 14th February 2003 at 19:06. |
|
14th February 2003, 19:20 | #13 | Link |
retired
Join Date: Jun 2002
Location: hollywood
Posts: 1,013
|
@Tom
To make sure I did another short test encode with the script below, using Nic's Feb.6 build and quant2. All options default except quantization type. ------------------------------------------------------- mpeg2source("C:\NATURAL_BORN_KILLERS\VIDEO_TS\NBK.d2v") crop(4,12,-4,-12) Trim(2000,4000) BilinearResize(512,272) ------------------------------------------------------- SoftMatrix (mpeg-custom ): 12378kb h263: 13582kb The results are the same -> very visible edge artifacts and mosquito noise with SoftMatrix. screenshot from the softmatrix encode: Last edited by iago; 14th February 2003 at 19:29. |
14th February 2003, 20:46 | #16 | Link |
Registered User
Join Date: Feb 2002
Posts: 1,195
|
hi,
the cutoff matrix give better result. cutoff => 9.86 Mo the result was near the h263 matrix could u explain exactely which are cutoff (but perhaps the explaination is too technical). but to sum up i like the result i think i will add this matrix in autodub , unhappyly i was a bit sad it was impossible to modulate 2 cutom matrix (exemple cutoff for first pass and modulate with the mpeg matrix but i think it was hard to select which quantlevel must be switched beetween the 2 matrix.) but the result was already good . PS: i have test your first matrix with an anime and the result was not very good . thank u for your work Bye.
__________________
AutoDub v1.8 : Divx3/4/5 & Xvid Video codec and .OGG/.MP3/.AC3/.WMA audio codec. AutoRV10 v1.0 : Use RealVideo 10 Codec and support 2 Audio Streams and Subtitles. Last edited by Dark-Cracker; 14th February 2003 at 20:49. |
14th February 2003, 21:07 | #17 | Link |
Registered User
Join Date: Oct 2001
Location: Gainesville FL USA
Posts: 2,092
|
It does appear that overly filtering this way makes it hard to get sharp edges like in text overlays or some anime. It's funny I haven't noticed the same thing as much with DctFilter.
I'd guess this would be especially noticeable at high bit rates with quant 2 so this type of filtering might possibly work better as the low side of some modulated quant process. In any event, the real test is to compare quality at the same bit rates. All quant 2 encodes are not equal if you are changing the matrices. I've been comparing these recently just using a 1 pass CBR just to see what I can get away with. I only tested it on video images, though both at high and low quants and bit rates. Though, as I mentioned above, the first one I posted (SoftMatrix) was intended to be a pretty extreme example. It throws away well over half of the information in the DCT output. But it also gives the most improvement in file size, or quality at the same low sizes. The best way to see the matrices is just to load them. It's too late now but if you save your existing default custom matrix first you can bring up 2 copies of Vdub and compare them. The values where 255 (previously 99) is plugged in are those higher frequency diagonals I intended to erase. Basically the further you are from the top left corner the higher the spacial frequency it is representing. - Tom |
15th February 2003, 03:46 | #18 | Link |
Registered User
Join Date: Oct 2002
Location: Portugal
Posts: 1,182
|
It would be good if someone test these quant matrices against H.263:
CBR 1-pass: Filesize, PSNR, detail loss perception; 2-pass: PSNR, detail loss perception; From what I've seen so far, it seems we can do smaller 1-pass than H.263 with some matrices at a similar but worst quality, mostly on edges. I also got the impression that higher H.263 quantizers can go unnoticed than with custom MPEG quantizers, so 2-pass encoding would benefit from H.263 on edges and high-quant handling. Bilu |
15th February 2003, 05:29 | #19 | Link |
Registered User
Join Date: Oct 2001
Location: Gainesville FL USA
Posts: 2,092
|
I had figured the advantage would go to these quant tables as the quant went UP and bit rate went down.
At low quants and high bit rates H263 ought to be better because it discards less info. And at very high bit rates you would think the MPEG quant would do best, for the same reason. I think that's theory Koepi used in his modulate quants. And I don't completely trust PSNR here because I'm purposely discarding information. But I can't offer anything better besides just eyeballs. My main justification for these is really that you may already be doing some filtering anyway that will remove high frequency info. And, if so, it's nice to do it in a way that compresses better and doesn't take extra processing power. But certainly if you have a very clean highly detailed clip that you would like to compress at quant=1 (or even 2) then you would not want to filter it this way. It's just been my own experience with my HDTV caps that the very highest frequency details are probably grain and mosquito noise anyway and I can live without them. But I agree the edge artifacts on SoftMatrix are a pain. - Tom |
15th February 2003, 16:44 | #20 | Link | |
Registered User
Join Date: Mar 2002
Posts: 1,075
|
Quote:
You would simply have to change the set_intra_matrix & set_inter_matrix routines. When an entry is 0 they should change it to 255 for the normal matrix entry and 0 for the fixed matrix entry (also the C version of the quantization routines would have to be updated to use the fixed matrices to implement division rather than use normal division like it does now ... this is a good thing anyway, C and assembly routines should use the same algorithm as much as possible IMO). |
|
Thread Tools | Search this Thread |
Display Modes | |
|
|