How to prevent grain induced block noise in uniform areas?

[adoniscik]

I am trying to get a 1MBps video with a smooth sky and a reasonable encoding rate (over 10 fps), but I realize this may not be possible. If not, I want to know how close I can get.

I read the archives voraciously, yet I could not solve the problem. Some of the suggestions contradict each other! Some people say block noise is due to bit starvation, so noise should be added. Others say it is a result of noise, and should be filtered out. What is a poor soul to do?

Since there was so much grain to begin with I felt no need to use the dithering approach. That leaves the noise reduction approach, of which I tried the following Avisynth 2.5 filters to no avail, used after de-interlacing with tomsmocomp(1,5,1):

• mpeg2source with the cpu=6 parameter.
• Convolution3d(preset="movieLQ") (as well as other parameters)
• peachsmoother(NoiseReduction=100)
• smoothhiq(7,20,20,256,0)
• VagueDenoiser(threshold=0.8,method=1,nsteps=6,chroma=true)

The only thing that remotely helped was enabling "Film Effect" in the XVID decoder.

Xvid 1.0 parameters:

• AS @ L5, Two-pass @ 1000kbps
• "Ultra High" motion precision search
• "Mode decision" VHQ mode
• Default quantizer restrictions: 1,31,1,31,1,31
• Trellision quantization
• MPEG quantizer (but I tried 'em all)

[Asmodian]

Try with no trellision quantization. Also a quant range of 2-31 and the h263 or HVS Best matrix might help.

EDIT: Do you just use standard B frame settings?

[*.mp4 guy]

try these settings:

custom mpg matrix sooulhuntersv5:  " "$ "$0 "$08 "$08@ "$08@€  $ $* $*0 $*08 $*08@ $*08@` $*08@`€

gmc
interlaced
qpel

default bvops
packed bitstream
closed gov

chroma optimizer enabled

motion search6
vhq4
use chroma motion
I frame interval 300

min quant 2 for all frames
trellis quant

to use the custom matrix just copy paste it into a txt file and open the txt file under the load custom matrix section.

[iago]

Blockiness on smooth surfaces, which contain little detail... A widely-discussed and never ending problem. Filtering out the grain only makes things worse in such situations.

Do not filter out the grain and use a matrix that helps keep details. (Since you say the film-effect -adding grain- during playback doesn't disturb you.) If high quantizers are used in that scene, use "zones" feature to assign lower quants to that particular section.

[Manao]

adoniscik : blocks are indeed due to overquantization of relatively smooth areas. This overquantization happens when you don't have enough bits to properly encode a frame. It is mainly visible in smooth area, because blocks are more visible in those areas.

So, a way to fight it is to make those areas less smooth, that's why some people were advising to add noise. But there is a counterpart : if you add noise, you're making the movie less compressible, so you'll lower the overall quality of the movie. The trick is to add enough noise to make the blocks almost disappear, and yet retain enough quality for the rest of the movie.

However, I would tend to use postprocessing to get rid of blocks. I don't like the idea of making the movie less compressible ( but that's a personnal opinion ).

Now, for XviD's settings, I don't know whether you used b-frames or not, but you should use them. As Asmodian say, you should try without Treillis quantization, because one of its effect is to discard useless DCT coefficients, which alas creates such blocks. You should definitely try a custom matrix ( I like HVS-Best )

[Teegedeck]

I do agree with iago and Manao. Though I'm not sure about disabling Trellis. Because Trellis has very good effects that I wouldn't want to miss. For example it gives a lower filesize while contours seem to remain more accurate with it than witout it, and in most cases it seems to prevent more quantization errors than it creates...

As iago said, first try to quantize the problematic zone less, use hvs-best as Manao said. If that doesn't help, add noise with that avisynth filter, err, I believe its name was "blockbuster", but only for that zone.

[Lord_KiRon]

adoniscik - are you by any chance try to play your clip on MTK based hardware MPEG4 player ? - This players are known to produce a lot of blocks when displaying uniform areas , when on PC they are noticable only if you set high resolution .

[adoniscik]

No, I am using the standard PC players; BSPlayer, MPC, and Media Player. I did use B-frames, at the Xvid 1.0 standard settings.

I ended up using noise reduction and enabling "film effect" in the decoder, but keep the suggestions coming. I am sure someone will benefit from them. Out of curiosity, does this problem exist with wavelet compression?

[Prettz]

The best solution is to completely and totally eliminate all noise in the still areas. If you can do this, you will (obviously) eliminate the problem. However, usually this is not possible, and in some cases attempting to remove the noise just makes the compressed result look even worse than before (especially when you try to remove the noise with Convolution3D).

In cases where you can't eliminate the noise, you should just do some light filtering (with filters like FluxSmooth and Undot) and use zones to up the bitrate, lower the quantizers, and reduce the number of bframes used. At least, this is the solution that has worked best for me in the past.

[Manao]

The best solution is to completely and totally eliminate all noise in the still areas. If you can do this, you will (obviously) eliminate the problem.

No. Doing so, you'll end up with a smooth gradient, which will be overquantized by XviD, because there isn't enough detail in it. Overquantization will break the smooth gradient ( in a sort of step-function ( if that word exists ) ), hence creating blocks. Such denoising will only prevent them from moving.

[eb]

adoniscik,

I processed your sample, you can find it on

ftp://www.eb.enterpol.pl
user: www.eb.enterpol.pl
password : eb

name of the sample ss1++.avi

eb

[adoniscik]

I downloaded your version. Thank you for the effort. The blocking is gone, but I notice ghosting, presumably due to aggressive temporal smoothing. Do you see it too? Judging by the 4CC, I think you used 3ivx rather than xvid.

[eb]

Yes 3ivx was used.
But most weired in your sample is shaking picture so I used deshaker plugin, then smoother,denoiser and again smoother, all this in the second pass of deshaker.
eb

[adoniscik]

Is that Gunnar Thalin's deshaker? I would imagine stabilizing affects compressibility because shaking makes motion estimation more difficult. I will investigate this. What is the performance penalty?

[lordadmira]

Yeah, it's a common problem u have there. But if you ask me trying to get 1 Mbps is pushing it for that frame size. Nevertheless I gave a go at it, I reused most of the settings I use for encoding my Inuyasha eps. I got a 504kB clip that looks much better than eb's attempt (overall). He used Quant 1 which is insane. I uploaded the config file too so you can check out the settings.

The problem with fine gradients like that is like Manao said, it's a problem with the dynamic range. However I don't see that problem in ur clip. The sky is chaotic enough to encode fine. It's the *really* fine gradients that have the potential to break the codec. Any blockiness u see is do to too low a bitrate. My encode went to about 820 kbps, 1000 is too high for this particular scene. I did a Q=2 encode and it only came out at under 1300 kbps. There is some choppiness to the sky (choppiness is diff from blockiness), that's normal for this codec for the dynamic range considerations already discussed. This is because the codec works by forcing square pegs into round holes, there's limitations to what that can achieve. Even Q=1 in my tests gave minor choppiness. To get any better u need custom matrixes. That's a trial and error process. The default matrix is limited in what it can do. I used Virtual Dub Mod and Smart Deinterlacer. The other filters are built in.

http://w3.goodnews.net/~wagnerc/adonisck.vcf

http://w3.goodnews.net/~wagnerc/adonisck[Xvid_1000].avi

LA

[lordadmira]

Originally posted by Manao
No. Doing so, you'll end up with a smooth gradient, which will be overquantized by XviD, because there isn't enough detail in it. Overquantization will break the smooth gradient ( in a sort of step-function ( if that word exists ) ), hence creating blocks. Such denoising will only prevent them from moving.

Eh, sort of. What we call a fine gradient is an area that has a small dynamic range. It's a range and pattern that doesn't correspond well to any of the quant matrix's transforms. So the codec has to force the area to take on one of set of bad matches. The result is choppiness. Blockiness is when the edge patterns of blocks don't line up/match up. The codec is optimized to handle well a certain range of patterns and gradients that occur in natural scenes. Fine gradients break that at one end of the spectrum and anime breaks it at the other end. Solid color -> black line -> solid color is not something it likes to deal with. It's the same phenomenon that causes the codec to go into rate control anarchy when it starts using quants 1's and 2's. Math might tell u that 1.52987 is the right choice, but sorry! U have to pick 1 or 2. Same with the fine gradients.

@iago
I don't think the amount of noise makes any difference to the dynamic range problem. Because although the noise will expand the absolute range, it will not change the average range. This is why temporal-spatial smoothers are able to filter out noise in the first place. So adding noise will just add to the blockiness problem since bits have to be spent to handle that large absolute range and do nothing for the choppiness problem which is one of average dynamic range.

LA

[Manao]

lordadmira : I think we're speaking of the same thing. With low quant, you can approximate very well ( to a certain extent ) all the values taken by DCT coefficients, while at higher quants, a stronger rounding will happen. When you round too much the coefficient that represent the mean value of the block ( I don't remember whether it is DC or AC, let's call it DC ), you obtain the block effect : DC coefficients will differ from one block to another, and continuity won't be brought back by other coefficients, because they are even more quantized.

[lordadmira]

Heheh, yeah, choosing the right words gets hard in these esoteric discussions. ^_^ U need to be a damn English major sometimes...

[manono]

adoniscik-

You've got a bad PAL to NTSC DVD there. Part of your problem stems from the fact that you're just deinterlacing it (and filtering the hell out of it) and keeping it at 29.97fps. If you were to use the KernelBob/RePAL combo, it'll become 24.975fps, distribute the available bits among fewer frames, have higher quality with a lower average quant for the same file size, and generally give you a much better looking result.

[eb]

lordadmira wrote

I got a 504kB clip that looks much better than eb's attempt (overall). He used Quant 1 which is insane.

Yes you are right, I used this to make quick assessment what max bitrate is needed to not to loose details, then I forget about this, jumping to the deshaking problem.
And I must underline that the prime target of my post was to take care of adoniscik to the shaking problem.

lordadmira:

Heheh, yeah, choosing the right words gets hard in these esoteric discussions. ^_^ U need to be a damn English major sometimes...

so what I can say with my crappy English

eb