Influence of denoise filter on the FPS of an Encoder

[Butcho]

Hello,

I've made some tests with denoising filters, see this
http://imageshack.com/a/img823/8570/4x7m.png

I've measured the Bitrates at constant QP values. The FPS values are the ones achieved by the encoder. It's the x264. I've alse made these tests with MPEG-2 and HEVC and the results are comparable. I wonder why the FPS achieved by encoding the filtered videostreams is constantly higher in comparison with encoding the original one, while the effect of having a lower bitrate decreases with a higher QP.

I think the approaching of the bitrates is caused by having less DC coefficients and so the influence of the denoising decreases because of the "internal lowpass" of an encoder.

But I cannot really imagine the effect on the speed of the encoder. It maybe has something to do with the prediction. Because of less noise there's a better prediction and so the RLC/VLC is getting better or more effective :confused: . Does anyone has experience in that?

PS: in the picture "3x3 Rechteck" = 3x3 Boxfilter, "Datenrate" = Bitrate
sorry for not having everything in english, but the meaning should be clear

[LoRd_MuldeR]

Noise is inherently "difficult" to encode, because video compression is all about discarding redundant data - but noise is random by definition.

This means you will end up with a much higher bitrate for a noisy video, compared to a denoised version - at least when encoding with a constant quantizer. And that's consistent with your results.

A higher bitrate means that more bits will need to be processed at the entropy coding stage (e.g. CABAC). That alone might explain a notable speed difference.

Another thing that comes to my mind: There are iterative algorithms going on, most notably motion estimation. In the noisy video it will probably take longer until the algorithm converges.

[Butcho]

Thanks for the reply.
I agree when having a higher bitrate it should take longer to encode so the fps of the encoder is lower. With a QP of 50 the bitrates are nearly equal, but the increase of the fps is still as high as with a QP of 20.

I also thought about motion estimation but I dont exactly know the functionality. Until now I thought there's a specific search area and after trying all possibilities the best match is taken. But if you say its iterative, it might stop when there's a good match found. Since motion estimation has a huge impact on the computational cost, this might be a good explanation.

edit:I didnt change the motion estimation settings, so its the default value of x264, which is hexagon

[LoRd_MuldeR]

Thanks for the reply.
I agree when having a higher bitrate it should take longer to encode so the fps of the encoder is lower. With a QP of 50 the bitrates are nearly equal, but the increase of the fps is still as high as with a QP of 20.

It's not surprising that all bitrates go down at higher QP.

But maybe you should better plot the ratios of the bitrates (e.g. divide them all trough the "original" bitrate), rather than plotting the absolute bitrates, so we can see how much percent of bits you save at each QP.

Anyway, with QP's as high as 50, there isn't much left of the video anyway, so the noise probably doesn't make much of a difference then.

I also thought about motion estimation but I dont exactly know the functionality. Until now I thought there's a specific search area and after trying all possibilities the best match is taken. But if you say its iterative, it might stop when there's a good match found. Since motion estimation has a huge impact on the computational cost, this might be a good explanation.

As far as I know, H.264 predicts the motion vector from the previous frames. And only the difference (offset) between the predicted motion vector and the "best" one is coded in the bitstream. How the encoder decides what the "best" motion vector is, that is totally up to the encoder. But a "smart" algorithm probably starts with an initial motion vector (the predicted one) and then refines the result iteratively - until convergence. An exhaustive search ("trying all possibilities") is very slow!

Adding noise to the image certainly doesn't aid to the goal of getting this done quickly.

[foxyshadis]

Without noise, there can definitely be multiple perfect or near-perfect matches, too, especially for less detailed areas, and as soon as it finds one it'll stop searching for more. Adding a coat of noise over everything makes a perfect match impossible. Because the search takes into account the tradeoff between coding the pointer and coding the residual, the closer the best match, the better. (That tradeoff is RDO in a nutshell, and how exhaustively it searches for the best is me and subme in x264.)

One artifact of this is that as you raise the QP, motion vectors get shorter, so you end up with worse matches. At QP 50, a distant perfect match can still cost more than coding a closer one and quantizing the hell out of it. C'est la vie.

The higher your QP, the more beneficial more exhaustive searches are. At 18 you might see negligible difference between hex/subme3 and tesa/subme11, at 50 it might be a relatively significant quality or bitrate difference to find the absolute best match. (Not saying you should do that, but it's something to keep in mind.)