Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 26th November 2007, 01:39   #61  |  Link
MfA
Registered User
 
Join Date: Mar 2002
Posts: 1,075
High frequencies can mean an edge, in which case you want a low quantizer ... or it can mean noise or texture, in which case you want a high quantizer. Also expecting the encoders to make sane choices all the time is probably way too optimistic.

I still think your efforts would be better spend in a place where the data you want is actually available ... try taking a look at DMPGdec, writing a drop in replacement for it's post processing is relatively easy, and the postprocess function already gets an array with the per block quantizer values.
MfA is offline   Reply With Quote
Old 26th November 2007, 21:13   #62  |  Link
redfordxx
Registered User
 
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 863
Quote:
Originally Posted by MfA View Post
High frequencies can mean an edge, in which case you want a low quantizer ... or it can mean noise or texture, in which case you want a high quantizer
Well, I don't know what do you mean by texture here, but speaking about noise...I am not interested in denoising at this moment, so if I am not mistaken: presence of noise=>high freq=>was low quant=>no blocking...





Quote:
Originally Posted by MfA View Post
writing a drop in replacement for it's post processing is relatively easy, and the postprocess function already gets an array with the per block quantizer values.
Confused...my english is probably not good enough;-)
redfordxx is offline   Reply With Quote
Old 26th November 2007, 21:17   #63  |  Link
MfA
Registered User
 
Join Date: Mar 2002
Posts: 1,075
Take DGMPGdec source code, replace postprocess with your own deblocking code, compile, run.
MfA is offline   Reply With Quote
Old 27th November 2007, 00:30   #64  |  Link
redfordxx
Registered User
 
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 863
Aaa, that's what writing a drop means... Well, how about only adding an option to the MPEG2Source to produce the mask of quantizers? I could do then whatever i want with that later...

Then I really like to have confirmed from someone, whether there are other codecs with variable quant within one frame than those which can be processed by DGMPGdec...
redfordxx is offline   Reply With Quote
Old 27th November 2007, 03:16   #65  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,554
Sure, practically any codec can use adaptive quantization, and they can be stored in most containers. Not all implementations do, but it's very common to find MPEG (1-4) and VC-1 with quants varying per-block.

You have to take into account the quant matrix, if you really want to figure out what quant was there before, because a 'high detail' matrix can cause a lot of blocking while also keeping a lot of noise/texture. (Texture means a pattern, though it doesn't have to be regular; here it means all of the useful non-edge information.) High quants with these matrices cause a lot of ringing/mosquito noise, instead of just removing high frequencies.

Last edited by foxyshadis; 27th November 2007 at 03:20.
foxyshadis is offline   Reply With Quote
Old 27th November 2007, 10:57   #66  |  Link
redfordxx
Registered User
 
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 863
Quote:
Originally Posted by foxyshadis View Post
Sure, practically any codec can use adaptive quantization, and they can be stored in most containers. Not all implementations do, but it's very common to find MPEG (1-4) and VC-1 with quants varying per-block.
interesting...I tested few codecs with show quants option in FFDShow and X264, XVID, MP42 had constant and MPG1, MPG2 had varying...


Quote:
Originally Posted by foxyshadis View Post
You have to take into account the quant matrix, if you really want to figure out what quant was there before, because a 'high detail' matrix can cause a lot of blocking while also keeping a lot of noise/texture. (Texture means a pattern, though it doesn't have to be regular; here it means all of the useful non-edge information.) High quants with these matrices cause a lot of ringing/mosquito noise, instead of just removing high frequencies.
OK... the point is, when I know quant and matrix for block, i can calculate error or compression information loss, right?
error_uv=quant*m_uv/2
error_uv=max error for frequency uv
m_uv=value in quant matrix
...at least for non AVC keyframes, right?
redfordxx is offline   Reply With Quote
Old 28th November 2007, 00:26   #67  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,554
Quote:
Originally Posted by redfordxx View Post
interesting...I tested few codecs with show quants option in FFDShow and X264, XVID, MP42 had constant and MPG1, MPG2 had varying...
If you never use the capabilities, then AQ won't be present, obviously. You can make your own by enabling that option in xvid, x264 (patched), mainconcept h.264, Nero, or various ffdshow codecs (in the 'Masking' option page).
foxyshadis is offline   Reply With Quote
Old 29th November 2007, 10:30   #68  |  Link
redfordxx
Registered User
 
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 863
Hi,
after reading some MPEG-2 docs and thinking over I got following (pls comment whether I am mistaken if you like):
- maybe I know what is pattern already. extreme example: when all freqs are veeeery strongly quantized except the last one, then on decoded frame the last frequency can dominate and we see it as 2D wave…thats the pattern…? This example does not happen because the coefs are increasing to higher freqs…but is that the principle.
- the amount of detail in decoded block does not say anything about the possible error=>blocking magnitude
- the amount of detail in decoded block does say about the possible damage when zeroing hi-freq coefs with DCTFilter on shifted video
- if I learn the quantizer from DGDecode and read the Qmatrix, I can calculate various values and averages of max or expected error…this could help for deblocking adaptability
- this calculation is same way possible for I frames or P frames (except the [0,0] coef?), I don't know how about B frames
- surprise: it is not that a frame DCT is either frame based or field based. My feeling is it can differ every macroblock. Some blocks are encoded progressive, some interlaced within one frame…
R*
redfordxx is offline   Reply With Quote
Old 1st December 2007, 01:09   #69  |  Link
redfordxx
Registered User
 
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 863
Hi, again begging for opinions:
I played with excel and simulated the codec process using MPEG quant matrix I obtained from DGDecode and random 8x8 sample blocks with following results (maybe I am discovering discovered thing but thats the risk:
I defined
Code:
TME(Q)= (theoretical maximal error for quantizer Q) = (value of pixel[0,0] if all values of freq coefs before inverse quantization equal 0.5)
example:
TME(1)=7
TME(2)=12
TME(3)=19
TME(4)=24
TME(5)=30
Maybe my calculation is not 100% MPEG compliant, but roughly so...
Now all calculated values can be formed as multiplication of this TME for desired quantizer.
Result looks like:
Code:
EA(Vert)=expected absolut value of average of absolute values of pixel errors on vertical edges
E(Vert)=expected absolut value of average of pixel errors on vertical edges
MA(Vert)=maximum absolut value of average of absolute values of pixel errors on vertical edges
M(Vert)=maximum absolut value of average of pixel errors on vertical edges
....and
E(Vert)=1.6%
E(Hor)=1.7%
M(Vert)=6.2%
M(Hor)=6.4%

EA(Vert)=7%
EA(Hor)=6.7%
MA(Vert)=14.1%
MA(Hor)=13.3%

M(Max8x8)=47%
E(Max8x8)=28%
and so on
so then, when processing the video:
Code:
...macroblock with quant 10 comes, I know:
TME(10)=60
E(Vert)=1.6%*60=0.96
E(Hor)=1.7%*60=1.02
E(Hor)=6.4%*60=3.84
..and even per pixel if needed
E(pixel[0,0])=6.9%*60=4.14
M(pixel[0,0])=24.61%*60=14.766
E(pixel[1,0])=8.65%*60=5.19
M(pixel[1,0])=29,28%*60=17.568
The problem of the theory can be here
Code:
E(Hor)*TME(40)=1.7%*242=4.114
M(Hor)*TME(40)=6.4%*242=15.488
M(Max8x8)*TME(40)=47%*242=113.74
...estimated maximal possible error is 113. Is it really possible in real world or in such cases encoder chooses lower quant?

Conclusion: I think, this can be together with with edge detection solid base for adaptive deblocking in my filter. However
1) I modelled the samples simply =INT(RAND()*256), which is not reality...in reality the samples would be correlated and the error will be smaler. How to model more real-like situation, any ideas?
2) I hope there are no errors like the 113 example or higher. I guess the encoder chooses lower quant but how to estimate this. Is there any SNR or block complexity/variance rule which can help?

Thanks for ideas.

R*
redfordxx is offline   Reply With Quote
Old 2nd December 2007, 02:00   #70  |  Link
redfordxx
Registered User
 
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 863
Step one finished: I have the quantizer mask from MPEG file from here;-)

Now...can anyone guess the distribution of the error values in non-intra block? Or at least some average...or something. What I think is, that the values are small compared to intra values but can be negative... but any better estimation?
redfordxx is offline   Reply With Quote
Old 3rd December 2007, 14:35   #71  |  Link
redfordxx
Registered User
 
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 863
Can you tell me opinion how to make weighted smoothing correctly?

If we have weight clip w and values clip x then (symbolically):

Code:
s="1 2 3 2 1" #weights of moving average
y=w*x
z=y.convolution(s)
ww=w.convolution(s)
output=z/ww
this is clear.

But how about smoothing using DCTFilter?
Code:
y=w*x
z=y.DCTFilter(1,1,1,1,0,0,0,0) #for example
#now which formula for ww is correct?
ww=w
ww=w.DCTFilter(1,1,1,1,0,0,0,0) 
ww=w.DCTFilter(1,0,0,0,0,0,0,0) #probably not
output=z/ww

Thank you.
redfordxx is offline   Reply With Quote
Old 12th December 2007, 17:52   #72  |  Link
redfordxx
Registered User
 
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 863
@Tobias (or whoever else can answer)

Once you mentioned, that for smoothing in my filter, quantization with the original matrix which was used to encode would be better than using DCTFilter.
I am thinking about implementation. I plan to read DGDecode quants, and DGIndex quantmatrixlog, so it is possible. But can you please explain the reasons, so I know whether it is worth the effort?

If I finish, expect improved MMX SmoothDeblock, so no more speed issues… so I wanna do it well.
redfordxx is offline   Reply With Quote
Old 13th December 2007, 16:18   #73  |  Link
708145
Professional Lemming
 
708145's Avatar
 
Join Date: Dec 2003
Location: Stuttgart, Germany
Posts: 359
Hi redfordxx,

good work. Keep up the speed

Quote:
Originally Posted by redfordxx View Post
Once you mentioned, that for smoothing in my filter, quantization with the original matrix which was used to encode would be better than using DCTFilter.
I am thinking about implementation. I plan to read DGDecode quants, and DGIndex quantmatrixlog, so it is possible. But can you please explain the reasons, so I know whether it is worth the effort?
If you the original matrix and original quant values you keep the same detail that was retained during encoding. If you use a "smoother" matrix it would blur too much.

Could you post performance figures for your MMX code once you get them? It would be great to use that for playback on HD resolutions

bis besser,
708145
__________________
projects page: ELDER, SmoothD, etc.
708145 is offline   Reply With Quote
Old 13th December 2007, 17:00   #74  |  Link
redfordxx
Registered User
 
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 863
Quote:
Originally Posted by 708145 View Post
If you the original matrix and original quant values you keep the same detail that was retained during encoding.
Even if used shifted?

There will be the option to smooth with larger matrices...I think 12x12 or 16x16 might be interesting.
How to apply the rule you mentioned on that? What should look a matrix bigger than 8x8 like to smooth similarily like a the known 8x8 matrix?
Seems to me that the down right corner could look similar...but I dont know

Quote:
Could you post performance figures for your MMX code once you get them?
Well there is still a way to go...
Quote:
It would be great to use that for playback on HD resolutions
The thing is, I intend to make this filter two pass:
* first pass, which would be very fast, will be done on a clip with showQ=true and saved (look at the link few posts above)
* second, the real deblocking will happen
redfordxx is offline   Reply With Quote
Old 13th December 2007, 17:51   #75  |  Link
Soulhunter
Bored...
 
Soulhunter's Avatar
 
Join Date: Apr 2003
Location: Unknown
Posts: 2,812
Quote:
Originally Posted by redfordxx View Post
The thing is, I intend to make this filter two pass:
* first pass, which would be very fast, will be done on a clip with showQ=true and saved (look at the link few posts above)
* second, the real deblocking will happen
Well, for realtime processing the 2 steps could be interleaved, no?

Bwt, good to see some movement in here... :]


Thx n' Bye
__________________

Visit my IRC channel
Soulhunter is offline   Reply With Quote
Old 14th December 2007, 19:35   #76  |  Link
redfordxx
Registered User
 
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 863
Quote:
Originally Posted by 708145 View Post
If you the original matrix and original quant values you keep the same detail that was retained during encoding. If you use a "smoother" matrix it would blur too much.
I still don't see it:
Whatever frequencieds the matrix and quantizer quantize, it always results in high frequencies in shifted block. And these high frequencies should be partially/fully removed. Or it is different?
redfordxx is offline   Reply With Quote
Old 15th December 2007, 22:10   #77  |  Link
708145
Professional Lemming
 
708145's Avatar
 
Join Date: Dec 2003
Location: Stuttgart, Germany
Posts: 359
Quote:
Originally Posted by redfordxx View Post
Even if used shifted?
There will be the option to smooth with larger matrices...I think 12x12 or 16x16 might be interesting.
How to apply the rule you mentioned on that? What should look a matrix bigger than 8x8 like to smooth similarily like a the known 8x8 matrix?
Seems to me that the down right corner could look similar...but I dont know
Well this is a tough question as I have not tried that. But guessing from the fact that larger blocks tend to result in more ringing, but the ringing averages out with the shifting approach I have truely no clue

Quote:
Originally Posted by redfordxx View Post
I still don't see it:
Whatever frequencieds the matrix and quantizer quantize, it always results in high frequencies in shifted block. And these high frequencies should be partially/fully removed. Or it is different?
Those high frequencies that define sharp edges have to be kept. And those frequences will be treated in the same way as by the original matrix, so the look and feel stays the same.

bis besser,
708145
__________________
projects page: ELDER, SmoothD, etc.
708145 is offline   Reply With Quote
Old 15th December 2007, 22:14   #78  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,554
If you remove all or most high frequencies, you may as well just use Deblock() or fft3dfilter(). In fact I can't really find any way around the inevitable detail-or-noise aspect of ringing, except in special conditions like anime.
foxyshadis is offline   Reply With Quote
Old 15th December 2007, 22:42   #79  |  Link
MfA
Registered User
 
Join Date: Mar 2002
Posts: 1,075
Ringing is a harder problem, but deblocking doesn't need to do damage to detail (although the approach chosen here isn't good in that respect).
MfA is offline   Reply With Quote
Old 16th December 2007, 05:52   #80  |  Link
redfordxx
Registered User
 
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 863
Quote:
Originally Posted by 708145 View Post
Those high frequencies that define sharp edges have to be kept. And those frequences will be treated in the same way as by the original matrix, so the look and feel stays the same.
I think we are talking about different things
Code:
Original:                                     Highly quantized - blocking:
o              |                                                                           
  o            |                                             |                              
    o          |                              o o o o        |                              
      o        |                                      o o o o|                              
        o      |                                             |                              
          o    |                                             |                              
            o  |                                             |                              
              o|                                             |                              
               |o                                            |                              
               |  o                                          |                              
               |    o                                        |                              
               |      o                                      |                              
               |        o                                    |o o o o        
               |          o                                  |         o o o o
               |            o                                |                              
               |              o                              |                              
                                        high frquencies here ^
Code:
Removed hi freq on shifted block:
               |                              
               |                              
               |                              
o o o o        |                              
        o      |                              
          o o  |                              
              o|                              
               |                              
               |                              
               |o                              
               |  o o                           
               |      o                       
               |        o o o o               
               |                              
               |                              
               |                              
       | shifted block |
redfordxx is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 15:37.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, vBulletin Solutions Inc.