Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 ASP

Reply
 
Thread Tools Search this Thread Display Modes
Old 11th February 2003, 23:58   #1  |  Link
trbarry
Registered User
 
trbarry's Avatar
 
Join Date: Oct 2001
Location: Gainesville FL USA
Posts: 2,092
Free filtering using custom quants?

When you have a noisy source or are going to encode at a very low bit rate sometimes it's better to filter the heck out of it first, removing high frequency components that will cause artifacts and be hard to compress.

It's come up a couple times here before that it might be possible to just use Xvid custom quant matrices to sort of do this for free, with no run time CPU cost. I asked about it a couple times last summer but it seemed custom quants were busted then.

But since then they've apparently been fixed and this morning I played with it for awhile.

For an extreme case I made quant matrices that had the value 99 in most all but the top left few positions (see attached file SoftMatrix.zip). Then I made a 1280x720p 1 pass CBR of the recent HDTV Daredevil trailer at only 2.5 mbps, usually too low for most HDTV. It actually came out pretty good, better than I had expected.

Can anyone tell me what max number I'm allowed to use in Xvid custom quant matrices? I just filled in 99 and it seemed to work but I don't see it documented anywhere. I want a number that just forces some DCT components to zero as much as possible.

- Tom

Last edited by trbarry; 12th February 2003 at 00:01.
trbarry is offline   Reply With Quote
Old 12th February 2003, 02:00   #2  |  Link
SirDavidGuy
Flying Attack Donkey
 
Join Date: Mar 2002
Posts: 159
Hmmm.... Don't remember exactly, but the XviD sources have the quant matrices stored as 1 byte, limiting the max quant to 255. This should probably make most values equal to zero, since most things which might be bigger than 510 are in the real upper-left of the DCT-domain array; most other things don't transform back to valid spatial-domain arrays.
__________________
"Bork Bork Bork"
SirDavidGuy is offline   Reply With Quote
Old 12th February 2003, 05:44   #3  |  Link
angelyote
The frumious Bandersnatch
 
angelyote's Avatar
 
Join Date: Sep 2002
Posts: 78
I'm sorry this doesn't really help Tom with his search for maximum values for quantums. I hope it adds to the useful input this thread will produce.

I'd seen some requests by people for information about how custom matrices work but I didn't see any information. Though they seem powerful and sound interesting I think they remain a mystery to a large percentage of the people here.

Here is a link I found that I thought really gave me a clear picture of what the arrays mean.
http://www.ece.purdue.edu/~ace/jpeg-tut/jpegtut1.html

Dave

EDIT: This post is a complete re-write. I thought the information would be handy as a link instead of a quote from one of its pages. My apologies to anyone who responded in it's original shorter form.

Last edited by angelyote; 12th February 2003 at 06:03.
angelyote is offline   Reply With Quote
Old 12th February 2003, 11:10   #4  |  Link
CavalloPazzo
Registered User
 
Join Date: Oct 2001
Posts: 80
I'm not sure this would work, but maybe I'm totally wrong...I hope my poor english is enough to explain what I want to say...
I read somewhere (probably in this forum) that for effect of rounding (to not propagate error) the values stored would be 0 1 0 1 0 1, ecc. and not always 0.
I've searched the post I'm talking about with no success. Maybe some of the developer should answer.

Last edited by CavalloPazzo; 12th February 2003 at 11:13.
CavalloPazzo is offline   Reply With Quote
Old 14th February 2003, 11:53   #5  |  Link
bilu
Registered User
 
bilu's Avatar
 
Join Date: Oct 2002
Location: Portugal
Posts: 1,182
@Tom

Maybe we can get some inspiration in this thread and
this thread ?


Bilu

Last edited by bilu; 14th February 2003 at 15:47.
bilu is offline   Reply With Quote
Old 14th February 2003, 16:16   #6  |  Link
Dark-Cracker
Registered User
 
Dark-Cracker's Avatar
 
Join Date: Feb 2002
Posts: 1,195
@trbarry

could i feel free to add your matrix in my tool ? (i have added support of 9 custom matrix based on the file made by reference divx).
__________________

AutoDub v1.8 : Divx3/4/5 & Xvid Video codec and .OGG/.MP3/.AC3/.WMA audio codec.
AutoRV10 v1.0 : Use RealVideo 10 Codec and support 2 Audio Streams and Subtitles.

Dark-Cracker is offline   Reply With Quote
Old 14th February 2003, 16:37   #7  |  Link
trbarry
Registered User
 
trbarry's Avatar
 
Join Date: Oct 2001
Location: Gainesville FL USA
Posts: 2,092
Quote:
Maybe we can get some inspiration in this thread and
this thread ?
bilu -

Thanks for the ref. I was following that thread at the time but Xvid then still had some bugs relating to custom matrices.

But I'm also diverging from that viewpoint in another way. Most of those custom filters are trying to find an optimum blend of how to smoothly increase the quant factor from low to high frequency components, and this is good.

But there is no free lunch. I'm instead proposing to decide up front how much high frequency detail we are willing to sacrifice (or think is noise and artifacts anyway) and not try to keep it at all. In the worst case this will lose detail on easy to compress still scence but avoids the artifacts that you get with quantizers over 20 or so.

I do this just by setting all the lower diagonals to 255. This will hopefully drive all/most of the values for those high frequences completely to zero in the DCT result. I'd even set them to 9999 if Xvid allowed it, but it doesn't seem to matter.

It would be neat if this could be used with modulated quants somehow.

Koepi - Is this possible?



Quote:
Dark- Cracker
could i feel free to add your matrix in my tool ? (i have added support of 9 custom matrix based on the file made by reference divx).

Dark-Cracker -

Yes, of course. But thanks for asking.

However I've also attached a slightly less extreme case (Filtered_Cutoff_Matrix) that I think works much better. I'm probably going to start using this one for many of my future encodes.

So you might want to try that one first. It's my best so far.

- Tom
trbarry is offline   Reply With Quote
Old 14th February 2003, 17:23   #8  |  Link
iago
retired
 
iago's Avatar
 
Join Date: Jun 2002
Location: hollywood
Posts: 1,013
@Tom

I'm waiting for the attachment to get approved, I'm really anxious to test the new matrix of yours.

The previous one gave me very visible edge artifacts in a test encode on Natural Born Killers using quant2[edited] .

(Btw, for some extreme cases, that is for some "mission impossible" encodes when aimed for 1CD , I have been getting really good results with DctFilter using some aggressive settings together with a combo of FluxSmooth/BilinearResize/cpu=6 (with Marc's mpeg2dec3.dll) in order to get rid of edge artifacts. Of course the result is really smooth, but it's at least free from blockiness and other artifacts, and for the time being this method seems to be the only solution for me when it comes to some movies that are otherwise "virtually impossible" to put on 1CD, such as Natural Born Killers .)

thanks for all your effort,
best regards,
iago

edit - but I must admit that I used b-frames as well .

Last edited by iago; 14th February 2003 at 17:38.
iago is offline   Reply With Quote
Old 14th February 2003, 17:36   #9  |  Link
trbarry
Registered User
 
trbarry's Avatar
 
Join Date: Oct 2001
Location: Gainesville FL USA
Posts: 2,092
"The previous one gave me very visible edge artifacts in a test encode on Natural Born Killers using quant2/h263."

I'm not sure I understand here. Did you have H263 selected for the quant type, or MPEG-custom? I though only the latter would actually use whatever you loaded for a custom matrix.

- Tom
trbarry is offline   Reply With Quote
Old 14th February 2003, 17:38   #10  |  Link
iago
retired
 
iago's Avatar
 
Join Date: Jun 2002
Location: hollywood
Posts: 1,013
@Tom,

Sorry, of course you are right, a terrible typo. Sure I used mpeg-custom loading your soft matrix with Nic's latest build. (I have corrected that mistake in the post above.)

Last edited by iago; 14th February 2003 at 17:42.
iago is offline   Reply With Quote
Old 14th February 2003, 17:51   #11  |  Link
Dark-Cracker
Registered User
 
Dark-Cracker's Avatar
 
Join Date: Feb 2002
Posts: 1,195
hi,

your matrix seems to be good for low bitrate , i have tested on "swordfish" i have lost some details but the final filesize was lower than h263 encode.

1pass, 10 Mo wanted, 640*272 , bitrate 650 kbps.

h263 => 9.85 Mo
softmatrix => 9.76 Mo

here some screenshot (screen vdub X 2)
http://www.eclipsedvd.firstream.net/...rix/1-h263.jpg
http://www.eclipsedvd.firstream.net/...softMatrix.jpg
http://www.eclipsedvd.firstream.net/...rix/3-h263.jpg
http://www.eclipsedvd.firstream.net/...softmatrix.jpg

u lose some details i think this matrix will be good for hight resolution (i think less block) or for anime.

i wait the moderator accept to test the next matrix, less strong.

[EDIT] no b-frame used only chromaM and search : 6, MinMax i : 2/4 MinMax p : 2/12, MinMax kf inter. 10/300 frm. [/EDIT]

Bye.
__________________

AutoDub v1.8 : Divx3/4/5 & Xvid Video codec and .OGG/.MP3/.AC3/.WMA audio codec.
AutoRV10 v1.0 : Use RealVideo 10 Codec and support 2 Audio Streams and Subtitles.


Last edited by Dark-Cracker; 14th February 2003 at 17:59.
Dark-Cracker is offline   Reply With Quote
Old 14th February 2003, 18:03   #12  |  Link
bilu
Registered User
 
bilu's Avatar
 
Join Date: Oct 2002
Location: Portugal
Posts: 1,182
Quote:
Originally posted by iago

(Btw, for some extreme cases, that is for some "mission impossible" encodes when aimed for 1CD , I have been getting really good results with DctFilter using some aggressive settings together with a combo of FluxSmooth/BilinearResize/cpu=6 (with Marc's mpeg2dec3.dll) in order to get rid of edge artifacts. Of course the result is really smooth, but it's at least free from blockiness and other artifacts, and for the time being this method seems to be the only solution for me when it comes to some movies that are otherwise "virtually impossible" to put on 1CD, such as Natural Born Killers .)

thanks for all your effort,
best regards,
iago

edit - but I must admit that I used b-frames as well . [/B]
In Avisynth 2.07 I've made it with

Unfilter(-5,-5)/FluxSmooth(5,5)/Unfilter(-5,-5)/cpu=0 and B-F (3/150/100)

also too smooth, but no MB visible.

Bilu

Last edited by bilu; 14th February 2003 at 18:06.
bilu is offline   Reply With Quote
Old 14th February 2003, 18:20   #13  |  Link
iago
retired
 
iago's Avatar
 
Join Date: Jun 2002
Location: hollywood
Posts: 1,013
@Tom

To make sure I did another short test encode with the script below, using Nic's Feb.6 build and quant2. All options default except quantization type.

-------------------------------------------------------
mpeg2source("C:\NATURAL_BORN_KILLERS\VIDEO_TS\NBK.d2v")
crop(4,12,-4,-12)
Trim(2000,4000)
BilinearResize(512,272)
-------------------------------------------------------

SoftMatrix (mpeg-custom ): 12378kb
h263: 13582kb

The results are the same -> very visible edge artifacts and mosquito noise with SoftMatrix.

screenshot from the softmatrix encode:

Last edited by iago; 14th February 2003 at 18:29.
iago is offline   Reply With Quote
Old 14th February 2003, 18:22   #14  |  Link
iago
retired
 
iago's Avatar
 
Join Date: Jun 2002
Location: hollywood
Posts: 1,013
screenshot from the h263 encode (same frame):

Last edited by iago; 14th February 2003 at 18:28.
iago is offline   Reply With Quote
Old 14th February 2003, 18:47   #15  |  Link
iago
retired
 
iago's Avatar
 
Join Date: Jun 2002
Location: hollywood
Posts: 1,013
However, it might well be a problem due to that particular build or due to the lower resolution used, etc. I don't know actually.

regards,
iago

Last edited by iago; 14th February 2003 at 19:46.
iago is offline   Reply With Quote
Old 14th February 2003, 19:46   #16  |  Link
Dark-Cracker
Registered User
 
Dark-Cracker's Avatar
 
Join Date: Feb 2002
Posts: 1,195
hi,

the cutoff matrix give better result.

cutoff => 9.86 Mo

the result was near the h263 matrix could u explain exactely which are cutoff (but perhaps the explaination is too technical). but to sum up i like the result i think i will add this matrix in autodub , unhappyly i was a bit sad it was impossible to modulate 2 cutom matrix (exemple cutoff for first pass and modulate with the mpeg matrix but i think it was hard to select which quantlevel must be switched beetween the 2 matrix.) but the result was already good .

PS: i have test your first matrix with an anime and the result was not very good .

thank u for your work

Bye.
__________________

AutoDub v1.8 : Divx3/4/5 & Xvid Video codec and .OGG/.MP3/.AC3/.WMA audio codec.
AutoRV10 v1.0 : Use RealVideo 10 Codec and support 2 Audio Streams and Subtitles.


Last edited by Dark-Cracker; 14th February 2003 at 19:49.
Dark-Cracker is offline   Reply With Quote
Old 14th February 2003, 20:07   #17  |  Link
trbarry
Registered User
 
trbarry's Avatar
 
Join Date: Oct 2001
Location: Gainesville FL USA
Posts: 2,092
It does appear that overly filtering this way makes it hard to get sharp edges like in text overlays or some anime. It's funny I haven't noticed the same thing as much with DctFilter.

I'd guess this would be especially noticeable at high bit rates with quant 2 so this type of filtering might possibly work better as the low side of some modulated quant process.

In any event, the real test is to compare quality at the same bit rates. All quant 2 encodes are not equal if you are changing the matrices. I've been comparing these recently just using a 1 pass CBR just to see what I can get away with.

I only tested it on video images, though both at high and low quants and bit rates.

Though, as I mentioned above, the first one I posted (SoftMatrix) was intended to be a pretty extreme example. It throws away well over half of the information in the DCT output. But it also gives the most improvement in file size, or quality at the same low sizes.

The best way to see the matrices is just to load them. It's too late now but if you save your existing default custom matrix first you can bring up 2 copies of Vdub and compare them. The values where 255 (previously 99) is plugged in are those higher frequency diagonals I intended to erase. Basically the further you are from the top left corner the higher the spacial frequency it is representing.

- Tom
trbarry is offline   Reply With Quote
Old 15th February 2003, 02:46   #18  |  Link
bilu
Registered User
 
bilu's Avatar
 
Join Date: Oct 2002
Location: Portugal
Posts: 1,182
It would be good if someone test these quant matrices against H.263:

CBR 1-pass: Filesize, PSNR, detail loss perception;
2-pass: PSNR, detail loss perception;

From what I've seen so far, it seems we can do smaller 1-pass than H.263 with some matrices at a similar but worst quality, mostly on edges.

I also got the impression that higher H.263 quantizers can go unnoticed than with custom MPEG quantizers, so 2-pass encoding would benefit from H.263 on edges and high-quant handling.


Bilu
bilu is offline   Reply With Quote
Old 15th February 2003, 04:29   #19  |  Link
trbarry
Registered User
 
trbarry's Avatar
 
Join Date: Oct 2001
Location: Gainesville FL USA
Posts: 2,092
I had figured the advantage would go to these quant tables as the quant went UP and bit rate went down.

At low quants and high bit rates H263 ought to be better because it discards less info. And at very high bit rates you would think the MPEG quant would do best, for the same reason. I think that's theory Koepi used in his modulate quants.

And I don't completely trust PSNR here because I'm purposely discarding information. But I can't offer anything better besides just eyeballs.

My main justification for these is really that you may already be doing some filtering anyway that will remove high frequency info. And, if so, it's nice to do it in a way that compresses better and doesn't take extra processing power.

But certainly if you have a very clean highly detailed clip that you would like to compress at quant=1 (or even 2) then you would not want to filter it this way. It's just been my own experience with my HDTV caps that the very highest frequency details are probably grain and mosquito noise anyway and I can live without them.

But I agree the edge artifacts on SoftMatrix are a pain.

- Tom
trbarry is offline   Reply With Quote
Old 15th February 2003, 15:44   #20  |  Link
MfA
Registered User
 
Join Date: Mar 2002
Posts: 1,075
Quote:
Originally posted by SirDavidGuy
Hmmm.... Don't remember exactly, but the XviD sources have the quant matrices stored as 1 byte, limiting the max quant to 255. This should probably make most values equal to zero, since most things which might be bigger than 510 are in the real upper-left of the DCT-domain array; most other things don't transform back to valid spatial-domain arrays.
Since 0 is not a valid value for a matrix entry we could use it to signify infinity. Since the assembly routines use multiplication to implement the division, division by infinity simply becomes multiplication by 0.

You would simply have to change the set_intra_matrix & set_inter_matrix routines. When an entry is 0 they should change it to 255 for the normal matrix entry and 0 for the fixed matrix entry (also the C version of the quantization routines would have to be updated to use the fixed matrices to implement division rather than use normal division like it does now ... this is a good thing anyway, C and assembly routines should use the same algorithm as much as possible IMO).
MfA is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 21:59.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.