What in the world is the inloop deblocker doing?

[Dark Shikari]

Encoded with:

--bframe 16 --b-rdo --bime --weightb --ref 6 --mixed-refs --b-pyramid --direct auto --deblock 1:1 --pass 3 --bitrate 660 --partitions "all" --8x8dct --me umh --threads 2 --thread-input --subme 7 --no-fast-pskip --trellis 2

Yes, its a low bitrate, but why, on the other hand, can I literally fill in the blocks in the output video using the paintbucket? Its as if the inloop deblocker isn't doing anything. Why?

http://i10.tinypic.com/67yfngj.png

[CruNcher]

Encoded with:

--bframe 16 --b-rdo --bime --weightb --ref 6 --mixed-refs --b-pyramid --direct auto --deblock 1:1 --pass 3 --bitrate 660 --partitions "all" --8x8dct --me umh --threads 2 --thread-input --subme 7 --no-fast-pskip --trellis 2

Yes, its a low bitrate, but why, on the other hand, can I literally fill in the blocks in the output video using the paintbucket? Its as if the inloop deblocker isn't doing anything. Why?


i think the reason is because x264 has no adaptive deblocking one that you can set by yourself for the quantrange so if blocks happening for a certain low quant range the inloop deblocker wouldn't see it and boom blocks.

[Dark Shikari]

i think the reason is because x264 has no adaptive deblocking one that you can set by yourself for the quantrange so if blocks happening for a certain range the inloop deblocker wouldn't see it and boom blocks.
I thought the inloop deblocker is always run for all appropriate blocks, regardless of quantizer? I don't really understand what you mean.

[*.mp4 guy]

The inloop deblocker is not run on "skipped blocks" reguardless of there quant; at the bitrate your using there is no forseable way for the quant to be so low that inloop is not used, so the majority of that frame must be made up of skipped/predicted blocks. Atleast thats the only explanation I can think of.

[Dark Shikari]

The inloop deblocker is not run on "skipped blocks" reguardless of there quant; at the bitrate your using there is no forseable way for the quant to be so low that inloop is not used, so the majority of that frame must be made up of skipped/predicted blocks. Atleast thats the only explanation I can think of.
Well then some advice: About 90% of this film is actually just fine encoded at this bitrate, with a bit of detail loss but nothing atrocious. However, in very dark areas of the frame the encoder just skips craptons of blocks, leading to blocking in a few specific parts of the film.

AQ, however, doesn't improve it; it actually increases the blocking. Why? The most noticable blocks are those that are moving, that is, there is a bit of background noise just enough to flip blocks from one color to the next between frames. So the blocks flicker between one color and another, leading to an extremely ugly appearance, even as high-detail non-flat areas of the frame look fine.

What should I do? I managed to make 300 look fine at a similar bitrate, but this is causing problems with all its black/blue areas.

[*.mp4 guy]

Raising AQ may just be causing more blocks to be skiped/predicted by putting X264 under a heavier bitrate crunch. You should try akupenguins aq+trellis+bframes fix in X264 rev667B, if you have not already, cef has a build available.

[Dark Shikari]

Raising AQ may just be causing more blocks to be skiped/predicted by putting X264 under a heavier bitrate crunch. You should try akupenguins aq+trellis+bframes fix in X264 rev667B, if you have not already, cef has a build available.
Using that build currently.

The problem might be in my noise reduction; the only way I can get acceptable quality at low bitrates is to completely kill all variation in non-moving portions of the video, and every denoising trick I've tried has failed with V for Vendetta, leaving just enough large-scale noise to screw up my encoding.

[*.mp4 guy]

If your willing to wait a while, it sounds like you should consider using an mvtools based denoising method, its very good at minimizing effects like this preemtively.

[akupenguin]

I can't tell for sure because the png is already converted to rgb, but most of those block edges are only +/- 1 luma. So h264's deblocker can't affect those blocks no matter how high the deblocking strength is, because there aren't any values in between for it to use. The only way any deblocker could help there is to increase the bitdepth, then deblock, then dither.

Leaving aside the issue of how to implement anything (e.g. I manually select qp and dct coefficients for each block), the possible ways to encode that content are:
* Blocky (as in your pic)
* Bandy (same sharp edges, just curves instead of squares)
* Dither, and reduce qp to lossless so that it keeps every single pixel of the dither.
* Increase the bitdepth. Then either dither on playback, or keep 10+ bits all the way to the display. This is inherently slow, and not supported by any(?) implementation other than JM... and even in JM you have to specifically enable support at compile time, because merely supporting 10bit makes 8bit slow.

The inloop deblocker is not run on "skipped blocks" reguardless of there quant
The reason the deblocker doesn't run on skipped blocks, is that in a skipped block nothing has changed since the last time that block was encoded as something other than skip, and the deblocker ran then.

[AlexW]

i think the reason is because x264 has no adaptive deblocking one that you can set by yourself for the quantrange so if blocks happening for a certain low quant range the inloop deblocker wouldn't see it and boom blocks.

do you mean adaptive deblocking offsets at the frame/slice level? because you can't actually use different deblocking offsets for different QPs within a frame/slice, all QPs have to use the same offsets.

[CruNcher]

yes qp level a matrix for every huh if this is right then Ateme implemented a non standard complaint thing way back the time ohhh hehe :P

[AlexW]

yes qp level a matrix for every huh if this is right then Ateme implemented a non standard complaint thing way back the time ohhh hehe :P

Ateme's custom deblocking matrices were probably just adapting deblocking offsets on the frame/slice level based on the average QP of the current frame/slice, i really doubt that they would have implemented something that wasn't in the standard ;)

[Gabriel_Bouvigne]

if this is right then Ateme implemented a non standard complaint thing way back the time
I don't think so...
Adaptive deblocking could mean adjusting deblocking per frame/slice, according to any combination of Qp/SliceType/InfoFrom1stPass/Wizzardry...

[Sharktooth]

Ateme's custom deblocking matrices were probably just adapting deblocking offsets on the frame/slice level based on the average QP of the current frame/slice, i really doubt that they would have implemented something that wasn't in the standard ;)
exactly

[addit]

So wouldn't one solution be to write an avisynth script which dithers low luma gradients on playback? I might try and give it a go myself if I have some free time.

[akupenguin]

Dither is a method for taking high bitdepth data and quantizing it to lower bitdepth. You can't just take some blocky content and dither it, the dithering filter has to know what it should look like (without the blocks). That's why I said deblock and dither, because there aren't any low luma gradients in the pic, they were replaced by blocks.
Simply adding gaussian noise has some effect, but it takes a lot more noise to hide artifacts than it takes dither to prevent artifacts. It's up to you whether you think the noise is worse than the blocks.

[Sharktooth]

... blockbuster in dither mode ...

[Didée]

... Deband in ffdshow ...

[Dark Shikari]

I can't tell for sure because the png is already converted to rgb, but most of those block edges are only +/- 1 luma. So h264's deblocker can't affect those blocks no matter how high the deblocking strength is, because there aren't any values in between for it to use. The only way any deblocker could help there is to increase the bitdepth, then deblock, then dither.

Leaving aside the issue of how to implement anything (e.g. I manually select qp and dct coefficients for each block), the possible ways to encode that content are:
* Blocky (as in your pic)
* Bandy (same sharp edges, just curves instead of squares)
* Dither, and reduce qp to lossless so that it keeps every single pixel of the dither.
* Increase the bitdepth. Then either dither on playback, or keep 10+ bits all the way to the display. This is inherently slow, and not supported by any(?) implementation other than JM... and even in JM you have to specifically enable support at compile time, because merely supporting 10bit makes 8bit slow.


The reason the deblocker doesn't run on skipped blocks, is that in a skipped block nothing has changed since the last time that block was encoded as something other than skip, and the deblocker ran then.
I want it to be either blocky or bandy, but without moving and fluctuating blocks. The blocks are not really noticable unless they move and blow up all over the place.

I used an MVTools-based denoiser already with both MVDegrain and FFT3DFilter, and I can't get anything strong enough to kill all temporal differences in static areas like I did with 300 (even though 300's grain seems much more difficult). I don't care about losing some detail; for one, I run the denoisers on the 1080p version, then downsize.

[Terranigma]

I guess you're talking about the de-block/noise script that *.mp4 guy, Didée and I concocted. Have you tried increasing the thsad and/or sigma? That helps as well. I have a clip here that needed the thsad strength raised to 800 because it was very grainy.
What should I do? I managed to make 300 look fine at a similar bitrate, but this is causing problems with all its black/blue areas.

I've experienced a similar issue like this as well basing my encode on a ratefactor of 22.0, and for black/blue areas, my source was coming out blocky. Heck, I even raised it to 18.0 and those black/blue areas still exhibited blocking, and because of this, i've just abandoned the clip.

[addit]

You're mistake was using x264... you should have used the fairchild codec :rolleyes:

[Dark Shikari]

I guess you're talking about the de-block/noise script that *.mp4 guy, Didée and I concocted. Have you tried increasing the thsad and/or sigma? That helps as well. I have a clip here that needed the thsad strength raised to 800 because it was very grainy.
Well I really changed your script because it was too slow (I lowered the motion search to one frame in each direction rather than two, and some other changes).

However, the problem is the gradfundb... it adds tons of noise, and when I take that out, the result looks pretty crappy overall, not nearly as good as my simple motion-compensated FFT3DFilter. :confused:

What is a good denoising filter that is very good at eliminating single-luma and single-chroma differences between frames that are low frequencies (i.e. not pixel-wide noise)?

[CruNcher]

I want it to be either blocky or bandy, but without moving and fluctuating blocks. The blocks are not really noticable unless they move and blow up all over the place.

Ahhh that's what you mean if it's the same problem i reported a year ago with relatively low compressed bitrate HD sample wich resulted in AlexW implementing --deadzones then i can say it's part of the look and feel of x264 since day 1 and at least Atemes implementation don't has this fluctuating block problem as i analyzed the partitions size i saw that the patern of those is changeing from frame to frame in x264 very randomly, in Atemes implementation it seems to be a not such a heavy difference so not so big partition changes, as those very chaotic ones in x264 hope you know what i mean tough :P. Pariscif.yuv at low bitrate is a very good example to see this :) but if i remember right this fluctuation especialy happened with noisy source i remember reporting the same issue to the Mainconcept/Elecard devs also :P.
AlexW do you remember it, is it maybe the same problem, from the situation it sounds so (low bitrate noise/grain result = block flicker) ?

[Terranigma]

CruNcher, do you plan on clearing some of your stored pm's to make room for new messages? :p

[Dark Shikari]

Ahhh that's what you mean if it's the same problem i reported a year ago with relatively low compressed bitrate HD sample wich resulted in AlexW implementing --deadzones then i can say it's part of the look and feel of x264 since day 1 and at least Atemes implementation don't has this fluctuating block problem as i analyzed the partitions size i saw that the patern of those is changeing from frame to frame in x264 very randomly, in Atemes implementation it seems to be a not such a heavy difference so not so big partition changes, as those very chaotic ones in x264 hope you know what i mean tough :P. Pariscif.yuv at low bitrate is a very good example to see this :) but if i remember right this fluctuation especialy happened with noisy source i remember reporting the same issue to the Mainconcept/Elecard devs also :P.
AlexW do you remember it, is it maybe the same problem, from the situation it sounds so (low bitrate noise/grain result = block flicker) ?
That does sound like it could be the problem.

I could always change the partition-finding function if I needed to and had an idea of what I had to do to it :)

Would turning off partitions altogether solve it? Or would the quality loss be too great?

The other possibility as stated earlier would be to find a filter that can completely kill all noise in motionless areas, eliminating the blockiness.

[Terranigma]

The other possibility as stated earlier would be to find a filter that can completely kill all noise in motionless areas, eliminating the blockiness.
Not really. I use that script I posted in the avisynth forums, and it does this, but yet I still get blocking in dark/blue areas.
Maybe it has something to do with the following.

1. Custom Matrices (Prestige CQM)
2. DCT Decimation with Trellis
3. Adaptive Quantization
4. B frames being used for those scenes when a P-Frame would be more suitable
5. Other.

[*.mp4 guy]

Ttempsmooth is a pretty good filter for locking down static areas, it does surprisingly well on mild low frequency fluctuations aswell.

[Dark Shikari]

Ttempsmooth is a pretty good filter for locking down static areas, it does surprisingly well on mild low frequency fluctuations aswell.
Actually, that worked surprisingly well!

Here's my script that I used that worked quite a bit better than my original and actually very slightly faster:
backward_vectors = source.MVAnalyse(isb = true,truemotion=true,delta = 1,idx = 1,pel=2)
forward_vectors = source.MVAnalyse(isb = false,truemotion=true,delta = 1,idx = 1,pel=2)
forward_compensation = source.MVFlow(forward_vectors, idx=1)
backward_compensation = source.MVFlow(backward_vectors, idx=1)
interleave(forward_compensation, source, backward_compensation)
fft3dgpu(precision=1,sigma=4,sharpen=0.7,bt=3)
TTempSmooth(maxr=1)
return selectevery(3,1).BlackmanResize(1280,720).TTempSmooth(maxr=7)

I'm trying a modification of this script to do a 5-frame-diameter window instead of a 3-frame window for the denoiser; its half the speed, of course. Its already quite a bit better than the original though and is much more watchable; I got 0.993 SSIM (!!!!) encoding a 2-minute scene at 660kbps, the average bitrate which was used in that scene my original encoding of the full film (which was 670kbps ABR, designed to fit on a CD)

Out of curiosity, how much would this script benefit from multithreading (and using fft3dfilter instead of fft3dgpu)? I only have a 7700 Go, so right now this script actually appears to be GPU-limited.

[Dark Shikari]

Here's a 10MB clip (http://www.mediafire.com/?cu3ibag5jno) of the resulting encode at 660kbps ABR using the aforementioned script.

The 5-frame diameter window caused artifacting so I dumped that idea. Right now I'm trying much stronger TTempSmooth options.

[Terranigma]

Talk about great quality with a 720p res and only 660kbps :eek:

[Dark Shikari]

Talk about great quality with a 720p res and only 660kbps :eek:I think people generally underestimate the effectiveness of x264 at extremely low bitrates :p

And note that encode used pretty low settings--just --subme 6 and --me hex and --ref 3 and so forth.

[Sharktooth]

impressive

[Didée]

that encode used pretty low settings--just --subme 6 and --me hex and --ref 3
Oh, didn't know about the --just parameter, have to try it. What does it do? :D

[Dark Shikari]

Oh, didn't know about the --just parameter, have to try it. What does it do? :D
http://myeve.eve-online.com/bitmaps/img/board_smilies/icon_lol.gif

[elguaxo]

Talk about great quality with a 720p res and only 660kbps :eek:

:eek::eek::eek:

[Dark Shikari]

:eek::eek::eek:
Better yet, try this (http://www.mediafire.com/?8ej3inttzza) re-encode with max x264 settings and slightly improved denoising.

Yes, that's right, almost 0.994 SSIM and over 50 average PSNR (!!!) as compared to the denoised source.

Oh the joys of denoising. :cool:

Edit: I just looked at the original footage again and damn that is some heavy grain; MVTools + TTempSmooth + FFT3DFilter is quite impressive I guess.

[ToS_Maverick]

nice result, but i really don't like the ugly, blocky gradients :(

isn't there anything we can do about it? AQ, film grain modeling...

[Dark Shikari]

nice result, but i really don't like the ugly, blocky gradients :(

isn't there anything we can do about it? AQ, film grain modeling...
The only ways to improve that, that I can think of, are:

a) FGM, to cover them up
b) 9-bit or 10-bit luma

The thing is though, the blocky, banded gradients are in the source itself. The grain covers it up. One would either have to use FGM, or better yet use a script that uses gradfundb and then massive denoising after that, and then passing THAT into an encoder that supports 9-bit or 10-bit luma.

[jeffy]

Dark Shikari, would you mind telling us what your source is?
I know it is the movie "V for Vendetta", but is this HD DVD or Blu-ray or HDTV or DVD? Thank you very much.

[Dark Shikari]

Dark Shikari, would you mind telling us what your source is?
I know it is the movie "V for Vendetta", but is this HD DVD or Blu-ray or HDTV or DVD? Thank you very much.
Its an HD-DVD (V for Vendetta isn't on Blu-Ray as far as I know). The video source is 1080p and the audio source is DD+.

[Terranigma]

The only ways to improve that, that I can think of, are:

a) FGM, to cover them up
b) 9-bit or 10-bit luma

The thing is though, the blocky, banded gradients are in the source itself. The grain covers it up. One would either have to use FGM, or better yet use a script that uses gradfundb and then massive denoising after that, and then passing THAT into an encoder that supports 9-bit or 10-bit luma.
1.9-bit/10-bit luma? How does one alter this?

2. What do you mean by FGM? Film Grain Matrix? I seen CruNcher mention the same thing here (http://forum.doom9.org/showpost.php?p=1034188&postcount=144) but he's always mocking us with the Ateme encoder. :p

3. You said you used maxed settings, but yet used 0 b-frames. Is there a good reason behind this?

4. Do you check your PM'S? :p

[Dark Shikari]

1.9-bit/10-bit luma? How does one alter this?

2. What do you mean by FGM? Film Grain Matrix? I seen CruNcher mention the same thing here (http://forum.doom9.org/showpost.php?p=1034188&postcount=144) but he's always mocking us with the Ateme encoder. :p

3. You said you used maxed settings, but yet used 0 b-frames. Is there a good reason behind this?

4. Do you check your PM'S? :p1. I believe H.264 supports 9-bit and 10-bit luma, which reduces banding. However, this not only require a bit more bitrate, but more importantly almost no consumer-level encoders support it except JM.

2. Film Grain Modelling.

3. 0 bframes? I forgot that on the commandline :eek::eek::eek::eek::eek::eek: Holy shit. I feel stupid. I had --b-pyramid and --weightb and --brdo but not --bframes! I am running the denoise script again (as I already deleted the losslessly encoded denoised clip) and will re-encode it with B-frames, oh god.

4. No, but I should :p

[Terranigma]

1. I believe H.264 supports 9-bit and 10-bit luma, which reduces banding. However, this not only require a bit more bitrate, but more importantly almost no consumer-level encoders support it except JM.

Ok, I downloaded the JM Reference Encoder from here (http://iphome.hhi.de/suehring/tml/download/) and compiled it. Now do you have a list of all the commands I can use via the lencod executable including changing the bit-depth of the luma? :p

[CruNcher]

FGM = Film Grain Modelling
FGT = Film Grain Technology

http://www.eetimes.com/showArticle.jhtml?articleID=59301182
actually it's very easy to understand what it is and the technology is not very new it's basicly a back working denoising/degraining algorithm the better the denoise/degrain results the better the regrain/renoise @ decoding :P (the denoiser/degrainer data is recorded as metadata in the bitstream, those the decoder reads and creates noise/grain from mathematical algorithms that matches the matrix of the metadata) in action it looks like this

http://www.yuvsoft.com/technologies/film_grain/index.html

but it costs time for HD can be alot of time @ encoding and the same for decoding so it needs alot of energy to recreate the noise eficiently and it's not only useable for H.264 it could be virtualy used for every Video Codec (but not without changeing the bitstream but sure you wouldn't need to record the matrix their you could put it in a file that the decoder reads) as for example Jomigo uses not a such advanced (i think they dont record the noise/grain but just denoise/degrain (encoder) and add artificial noise/grain (decoder) version in their Mpeg-4 ASP codec HDX4

basicly you can do the same as jomigos with X264 too just use a internal/external denoiser/degrainer --nr and later add the noise/grain @ decoding with ffdshow or mplayer for example ei voila FGM/FGT for poor people ;)

in both cases you do the same process Encoding (denoise/degrain) Decoding (regrain/renoise)

[Dark Shikari]

Ok, I downloaded the JM Reference Encoder from here (http://iphome.hhi.de/suehring/tml/download/) and compiled it. Now do you have a list of all the commands I can use via the lencod executable including changing the bit-depth of the luma? :pNo, as I've never used it before :p

[Terranigma]

nvm, there's an adobe .pdf document :p

[Sagittaire]

1) FGM is a official part of H264 specifications and for example HDDVD can play H264 stream with FGM. H264 can preserve very well grain at medium/high bitrate and FGM is only usefull for low or very low bitrate. FGM will be really usefull for HDDVD9 or BD9 1080p encoding for example.

2) For reduce the block flicking you can try to reduce the ratio between frame type and particulary ratio for P-B with long GOP encoding.

3) For reduce problem in dark area you can try a filtering only for Dark Zone (Dark Noise Filter) and use AQ with spatial complexity masking (lower quant for low complexity zone).

[CruNcher]

1) FGM is a official part of H264 specifications and for example HDDVD can play H264 stream with FGM. H264 can preserve very well grain at medium/high bitrate and FGM is only usefull for low or very low bitrate. FGM will be really usefull for HDDVD9 or BD9 1080p encoding for example.


Yes but you have to control the quantrange more restricted if it is to high grain gets lost very fast most times @ q30 with the flat matrix grain is completly lost and also take b-frames into account wich lose detail pretty fast too

[Dark Shikari]

New version up, this time with proper B-frames! :p

http://i11.tinypic.com/53z5ksm.png

Download Link (http://www.mediafire.com/?3lefdynqfna)

[ToS_Maverick]

because of such sick examples, i'd like a constant-ssim method for x264 :D

0.99 at average quant 21, normally you'd need a quant at about 15 to reach this...