--nr: how does it work?

[vmrsss]

Hi there,

Is there a technical (yet, possibly not the code itself!) explanation available of exactly how --nr works?

thanks

[nurbs]

Try akupenguin's posts here: http://forum.doom9.org/showthread.php?t=105433

[vmrsss]

Thanks. I read the paper and I think I understand it. I am still unclear however how this is implemented in x264 (which may be the same as in mencoder etc, but I am interested in x264).

Firstly, how does the commandline parameter determine the scaling of the DCT coefficient? (I ask this because it may be useful to estimate how much --nr to use in specific circumstances).

There is then the issue that allegedly --nr would only work well with 2-pass encodes. This is eg the kind of statements you can find on the net:


From: Phil Ehrens <phil <at> slug.org>
Subject: Re: Why use TOOLS/calcbpp.pl and scale ?
Date: 2006-03-02 17:10:22 GMT

Note that the behavior of nr is very different in 1-pass and 2-pass
modes. In 2-pass mode it is a brilliant thing. In 1-pass mode it is
an interesting "decimation with filtering" operation that sometimes
works wonderfully and sometimes creates strange artifacts.

Given the way it works, this made little sense to me. The best (only?) approach to an explanation I could find is in a message on the mencoder mailing list, whose relevant part reads like:


Lorren Merritt on Sep 7 2007

>> These numbers are for 2-pass encodes. If you are doing single-pass
>> encodes, hqdn3d will behave much more sanely than nr. But if you
>> are doing multi-pass, nr will be better on all live action. Anime
>> seems to really love hqdn3d, however.

Interesting. The only difference between 1 and 2pass I can think of that
might cause that is that 2pass allocates bits as a function of bits (i.e.
magnitude of quantized DCT coefficients), while 1pass allocates bits as a
function of SSD. DCT coefficients have passed through nr, while SSD
hasn't. So scenes with little content but much noise will be allocated
lots of bits in 1pass nr, but not in 2pass nr nor 1pass hqdn3d.

This essentially says that in 2-pass the allocation of bits to a frame takes into account the effect of --nr, in 1pass it doesn't. Which may lead to allocating to a frame more bits than necessary. Ehi, but wait a minute: if the extra bits are not used (eg because the quantizers are cut by --nr), they will be reallocated by the overflow compensation mechanism, will they not? So, why is that a deal-breaker...??

Would anybody help me understand what happens with --crf, which is my rate-control mechanism of choice? With --crf the complexity of the frame is computed approximatively via a fast motion estimation over a half-resolution version of the frame, using the SATD residuals and via the --crf commandline parameter. Again, this will not take into account the effect of --nr, so it will potentially overshoot the complexity of the frame, allocating lower QPs than otherwise required to achieve the required constant quality (right?) Then, because of the effect of --nr, such quality will be obtained with fewer bits than expected (right?), and this will have no consequence whatsoever on the rest of the encode (right?) and as far as I can see no effect at all on what --nr would do... (right?)

So, again, why would that affect, impair or limit in any way the working of --nr in any conceivable way? I just can't see it... If the statement is true, that can only relate to my first question above, eg that the amount of low-pass filter introduced by --nr is computed for each frame taking into account their estimated complexity (which is good in 2pass and poor in 1pass and --crf)

Thanks for your help

[Dark Shikari]

Noise reduction and 2-pass have nothing to do with each other. Ignore whoever said that.

[vmrsss]

Noise reduction and 2-pass have nothing to do with each other. Ignore whoever said that.

hmm, as i thought. thanks.

Do you happen to know how the parameter N in --nr N is yields a multiplicative factor for the DCT coefficients? Somewhere I read that N is meant to indicate the amount of noise in the picture (so it ought to be related to the amount of "information" in the frame before it can serve a useful purpose) It that were so, it might be more intuitive to have a number between 0 and 1...

[nm]

There is then the issue that allegedly --nr would only work well with 2-pass encodes. This is eg the kind of statements you can find on the net:


From: Phil Ehrens <phil <at> slug.org>
Subject: Re: Why use TOOLS/calcbpp.pl and scale ?
Date: 2006-03-02 17:10:22 GMT

Note that the behavior of nr is very different in 1-pass and 2-pass
modes. In 2-pass mode it is a brilliant thing. In 1-pass mode it is
an interesting "decimation with filtering" operation that sometimes
works wonderfully and sometimes creates strange artifacts.

There was a problem in nr implementation at that time, causing blocky artifacts in certain scenes. The effect was the same for both 2-pass and CRF. It was fixed on 2006-03-06 (http://git.videolan.org/gitweb.cgi?p=x264.git;a=commit;h=a34eec24d5caed8b43f2d1ecf7a0f36b9fe60189)

[vmrsss]

I would like to ask for your considerations on the following:

is there a tradeoff between raising --nr (or using a denoiser) and lowering --crf or, viceversa, lowering --nr and raising --crf ?

I mean, I could get to a same average bitrate by doing one of the two things above. How would the quality of the resulting encode compare?

Hope this makes sense, thanks.

[nm]

is there a tradeoff between raising --nr (or using a denoiser) and lowering --crf or, viceversa, lowering --nr and raising --crf ?

I mean, I could get to a same average bitrate by doing one of the two things above. How would the quality of the resulting encode compare?

Quality will depend on the denoiser used, and whether you want a clean video or something as close to the original as possible.

Even high quality denoisers will filter out some details such as fine rain, so for good results you might want to tweak the denoising strength manually for different scenes.

[vmrsss]

Thanks for your answer.

Quality will depend on the denoiser used, and whether you want a clean video or something as close to the original as possible.


OK, but also choosing too low a CRF will remove details (because it will result in too low a bitrate) will it not? If so, what kind of (visual) difference should one expect to notice between the two cases?