Fade-in ugly blocking problem, any progress?

[Terranigma]

wrong... completely. inloop is adaptive, so its not a problem at all. also you can change the adaptiveness bias with the strength and threshold options...

Useless as in, extra cpu usage for less sharper/pleasing results.

[Sharktooth]

and a blockfest without it... inloop is part of the h.264 specs for a reason...

[Terranigma]

and a blockfest without it...
Haha, not true. Maybe for you and your settings, but not mines. ;)

[Sharktooth]

are you sure? unless quants are really low three will be blocks more or less visible... but blocks.
you cant encode for example at quant 30 and have no blocks without inloop...

[Terranigma]

are you sure?
Yes I'm sure.

EDIT:
you cant encode for example at quant 30 and have no blocks without inloop...


That's true. I guess in-loop would help there, but i'm speaking from a range between 16-20; sometimes up to 22.

[Sharktooth]

in that range the deblock is not doing much... expecially with negative strength and/or threshold...IIRC @ Q15 it is always disabled (with 0:0).

[Dr.D]

One thing that I have noticed that hasn't been touched on is that the higher strength/threshold you set deblocking the encoder, the more often x264 purposefully blocks in large areas (usually flat) with the expectation of it being deblocked. With a low deblocking setting (say -3,-3) x264 usually seems to try to block in smaller, less obvious areas, likely in attempt to keep PSNR up and maintain perceptive quality. This may or may not be the case in the sample here considering that with a major lack of bitrate there is only so much x264 can do.

To sum it up, at least on anime content, if you are disabling in-loop deblocking in the decoder (for whatever reason) using a low deblock setting or even -no-deblock will usually give you a perceptually less-blocky output at the same bitrate in my experience when doing a multi-pass encode.
Thanks, it's exactly what I do normally (but tsss..., it's a big secret, I don't want to be electrocuted for that :)), and I like the results very much. Except fade-in spots.

[Dark Shikari]

Thanks, it's exactly what I do normally (but tsss..., it's a big secret, I don't want to be electrocuted for that :)), and I like the results very much. Except fade-in spots.Obviously it isn't what you do since you just said you disabled deblocking in the decoder... :rolleyes:

[cyberbeing]

Thanks, it's exactly what I do normally (but tsss..., it's a big secret, I don't want to be electrocuted for that :)), and I like the results very much. Except fade-in spots.

Well even though I said that, just to make sure you understand, you should still always avoid disabling in-loop deblocking in the decoder and control the deblocking strength/threshold in x264 when encoding.

The only time I disable deblocking in the decoder is when I'm trying to see where blocking is occurring and in effect where there will be detail loss / blurring when in-loop kicks in. Different deblocking strengths/thresholds will have blocking in different areas of a frame and sometimes different frames altogether. Since I like detailed encodes, my goal is to end up with minimal detail loss and not have any obvious blocking which is a bit of a see-saw. Sometimes this means higher in-loop and other times it means lower in-loop.

[akupenguin]

To sum it up, at least on anime content, if you are disabling in-loop deblocking in the decoder (for whatever reason) using a low deblock setting or even -no-deblock will usually give you a perceptually less-blocky output at the same bitrate in my experience when doing a multi-pass encode.
That's trivially true, but unrelated to your previous paragraph. If you encode with deblocking and decode without it, then you have encode-decoder desync, which is far uglier than any possible compression artifact. The weaker the deblocking, the less the desync in that case.

[cyberbeing]

That's trivially true, but unrelated to your previous paragraph. If you encode with deblocking and decode without it, then you have encode-decoder desync, which is far uglier than any possible compression artifact. The weaker the deblocking, the less the desync in that case.

This is the first time I've heard of encoder-decoder desync so I'm a bit confused by this. Are you saying that the blocking you see when disabling in-loop in the decoder doesn't actually exist in the stream? I don't see how that's possible but maybe I don't fully understand how h.264 in-loop deblocking works.

Can someone show me an example of what "encode-decoder desync" blocking looks like compared to "compression artifact" blocking?

[Dark Shikari]

I'm a bit confused by this. So are you saying that the blocking you see when disabling in-loop in the decoder doesn't actually exist in the stream?Correct.I don't see how that's possible but maybe I don't fully understand how h.264 in-loop deblocking works.

I'll give a simple example then:

Frame 0 is decoded, then deblocked. We'll call this output frame 0A.

Frame 1 is decoded, using Frame 0A as its reference.

If you turn off deblocking:

Frame 0 is decoded.

Frame 1 is decoded, using Frame 0 as its reference.

Since it uses a different frame as its reference, the output is going to be different--even if you deblock frame 1 and not frame 0, it'll look quite different than what the encoder intended.

Now imagine this error stacking up over 200 frames straight.

[LoRd_MuldeR]

I don't see how that's possible but maybe I don't fully understand how h.264 in-loop deblocking works.

In H.264 the deblocking filter is a mandatory part of the decoder. That's different from MPEG-2 or MPEG-4 ASP.
Hence the encoder relies on the decoder to do a proper deblocking. The encoder even references to deblocked frames!
So if you disable the deblocking in your H.264 decoder, you actually break the decoding and you'll get corrupted output.
You will not only see more blocking, you also get more serious artifacts that result from invalid reference frames...

[cyberbeing]

If I'm understanding you correctly, I'm assuming that means you can't judge compression artifacts in an encode by disabling in-loop in the decoder. How would you judge where compression artifacts are occurring and in turn what parts of a frame deblocking is being applied? When in-loop deblocking is turned on in the decoder it would be deblocking the blocks which defeats the purpose or identifying problem areas. Right?

[Dark Shikari]

When in-loop deblocking is turned on in the decoder it would be deblocking the blocks which defeats the purpose or identifying problem areas. Right?Huh? Deblocking is part of the encoding and decoding process--compression artifacts can only be measured when the entire encoding and decoding process is done correctly. If you do the process wrong, you'll get false results, obviously.

[cyberbeing]

While I understand what you're trying to say, I'm still under the belief that even with encoder-decoder desync, those areas are actually where blocking is occurring even if it may be showing up worse then the encoder intended. This seems to be supported by these area getting blurred when in-loop is turned back on in the decoder. Is this wrong?

[LoRd_MuldeR]

You might use h264visa (http://h264visa.com/index.html) or a similar tool to analyze your encode. It can show tree different "pictures" for each frame:
Predicted (result of reference+mc), Residual (difference between "predicted" and original), Pre-LoopFilter (result of predicted+residual, without deblocking) and Final (deblocked image)

[Dark Shikari]

While I understand what you're trying to say, I'm still under the belief that even with encoder-decoder desync, those areas are actually where blocking is occurring even if it may be worse then the encoder intended. This seems to be supported by these area getting blurred when in-loop is turned back on in the decoder. Is this wrong?The proper way to do this is to have a decoder that displays frames before they are deblocked, but still does the deblocking behind the scenes to avoid errors accumulating.

But I still don't understand your point--the entire purpose of the deblocking filter is to compensate for the weakness of block-based DCT formats by subtly blurring the edges of each block based on data from the stream. A block is not a "problem area" unless its problematic after the filter has had its way with it, too.

The entire purpose of the filter is so that the encoder doesn't have to worry about limiting blocking and can instead rely on the sane method of the loopfilter to deal with it. Also note that the purpose of the deblocking filter is first and foremost not visual; block edges tend to make motion compensation much less efficient, so resolving them increases coding efficiency even if you don't display the deblocked frame.

If you want to identify "problem areas", I'd say the best method is to take your eyes, watch the freaking video, and ask yourself what areas look problematic. Not try to make an analysis based on decoded frames that aren't supposed to exist.

[cyberbeing]

The proper way to do this is to have a decoder that displays frames before they are deblocked, but still does the deblocking behind the scenes to avoid errors accumulating.

Unfortunately there doesn't seem to be a free or affordable tool for this. h264visa sounds interesting but it's really expensive at $270 or $450 for the useful versions.


A block is not a "problem area" unless its problematic after the filter has had its way with it, too.
If a block appears over important detail in a frame, when de-blocked detail is lost. I view all blocking as problematic, just some less so then others.


If you want to identify "problem areas", I'd say the best method is to take your eyes, watch the freaking video, and ask yourself what areas look problematic. Not try to make an analysis based on decoded frames that aren't supposed to exist.
What I was doing was comparing the normal deblocked output, the output with deblocking disabled, and the source video to get a general idea for where in a frame blocking and hence detail loss was happening. It now seems this is not the best way to go about it so I guess I'll eliminate disabled decoder deblocking from the mix or at least keep in mind the decoding errors and take it's output for a grain of salt.

Thank you everybody for clearing up my misunderstanding.

[LoRd_MuldeR]

Unfortunately there doesn't seem to be a free or affordable tool for this. h264visa sounds interesting but it's really expensive at $270 or $450 for the useful versions.

You can test it for free. The unrestricted trial version works for 30 days and you can re-install as often as you like. No registration needed.

[Dark Shikari]

What I was doing was comparing the normal deblocked output, the output with deblocking disabled, and the source video to get a general idea for where in a frame blocking and hence detail loss was happening.Blocking and detail loss are completely orthogonal. In fact, most psychovisual optimizations I know of that bias in favor of more detailed images tend to increase blocking.

[cyberbeing]

you can re-install as often as you like
Oh, I thought they would have some sort of trial protection to prevent use even after a re-install, guess not. Thanks for the info LoRD_MuldeR.

Blocking and detail loss are completely orthogonal. In fact, most psychovisual optimizations I know of that bias in favor of sharper, more detailed images tend to increase blocking.
I don't mind a bit of blocking as long as it's not overly obvious, it's the blocking being blurred to hell with deblocking that can sometimes be an issue which is why I stick with negative in-loop settings. Hence higher in-loop in x264 leads to a softer, potentially less detailed output on an encode that has lots of blocking. All of a sudden your high detail source becomes a low detail x264 encode because of a high in-loop deblocking setting blurring the blocks which seem to tend to appear on detailed areas and sharp edges.

[LoRd_MuldeR]

Oh, I thought they would have some sort of trial protection to prevent use even after a re-install, guess not. Thanks for the info LoRD_MuldeR.

Obviously they do not, even though the product is quite expensive. I was surprised myself...

[Ranguvar]

I don't mind a bit of blocking as long as it's not overly obvious, it's the blocking being blurred to hell with deblocking that can sometimes be an issue which is why I stick with negative in-loop settings. Hence higher in-loop in x264 leads to a softer, potentially less detailed output on an encode that has lots of blocking. All of a sudden your high detail source becomes a low detail x264 encode because of a high in-loop deblocking setting blurring the blocks which seem to tend to appear on detailed areas and sharp edges.

Me, too - but I use *reasonably* low settings, like -2:-1. Killing deblocking, especially in only the encode or decode, or setting abnormally low values, is going to be really bad. And at such high quants, especially so.

A blur is better than a block, IMO. But I've been very surprised by the quality of the inloop deblocker. It's *far* better than post-processing deblockers. Blurs do not look like blurs. And besides, the only reason I am okay with some blockiness is because I'd rather have that than loose detail, like you. Well, x264 does a great job of being adaptive with its deblocking. Higher deblocking tends to do well at eliminating edge noise/dirt, and overall doesn't blur much at all.

All I'm saying is, do some tests. You may be surprised.

[cyberbeing]

I very seldom encode with what would be considered high quants where blocking can become a major issue, so that is not a problem for me. I have done plenty of tests of my own and I agree with you that x264 does have a good in-loop deblocker, but it still can blur detail if you're not careful even at relatively high bitrates. I'm of a bit different opinion where I usually prefer razor sharp and slightly blocky (no horrid macroblocking please) over soft and slightly blurry. If it becomes too blocky, I aim for lower quants and throw more bitrate at it considering I encode for archiving purposes.

[LoRd_MuldeR]

I agree with you that x264 does have a good in-loop deblocker

x264 does not "have" an in-loop deblocker. The inloop-deblocker is part of the H.264 specifications - all encoders and decoders use it in the same way...

The filter is applied to the vertical and horizontal edges of 4×4 chroma or luma blocks in a macroblock. The amount of filtering is measured by boundary strength (bS), which is decided by the quantiser parameter, coding modes of the adjacent blocks and the gradient of the sample cross the edge. Two thresholds (α and β) defined in the standard make decision on whether to perform filtering on the current boundary or not. α and β depend on the average QP of the two blocks adjacent to the edge: they increase with the increase of QP and vice versa. In the design of the deblocking filter, two controllable parameters α_offset and β_offset are added to provide a wider range of control of the filter along with QP. When α_offset and β_offset are set to zero, α and β depend on QP only. The values of α and β for any given QP have been chosen based on rate-distortion performance results.

[akupenguin]

Can someone show me an example of what "encode-decoder desync" blocking looks like compared to "compression artifact" blocking?
Desync artifacts don't necessarily look like blocks.

source (http://img140.imageshack.us/img140/7281/sourcefh5.png)
encode with deblock (http://img247.imageshack.us/img247/7414/deblockyt4.png)
encode without deblock (http://img247.imageshack.us/img247/2972/nodeblockhz0.png)
encode with deblock, decode without (http://img213.imageshack.us/img213/6914/desyncxm7.png)

[cyberbeing]

Wow, I've never seen anything like that before, must be lucky. Why do decoders even expose options to completely disable deblocking if it can cause such nasty side-effects? Performance shouldn't even be an excuse. Thanks akupenguin for the examples.

[woah!]

lets remember here he is giving this encode about 1500kbps on a 720 res fade-in source aswell... i would like to see the mainconcept result please, i bet it looks like ass aswell... also crf 25 for you isnt going to be the same for me if i use different settings, if i use crf 25 right now it gives me 3400kbps to play and of course looks way better... cant expect miracles mate...

try a 2pass fixed bitrate until you see a pleasing result and than adjust your crf to suit, i bet its way lower than 25...

heres my go at crf 28 which gives me a file the same size as your posted one : http://img524.imageshack.us/my.php?image=clipboard02ub3.png

[Quark.Fusion]

And yes, I usually only add a few IDR frames in problem areas like fade-in/fade-outs and long pans where x264 starts producing blocky/distorted P and B frames from lack of a good reference frame. I know that excessive use of a qpfile will degrade quality.
Maybe more refs and b-frames with --b-adapt 2 will help?

[cyberbeing]

Maybe more refs and b-frames with --b-adapt 2 will help?

It doesn't, already been tried on a few sources with no improvement. Once the current frame has nothing in common with the previous I frame quality starts degrading quickly and sometimes produces ugly blocking. Since x264 doesn't detect fade-in/fade-out or long pans as a scene change you have to add them manually.

[Dark Shikari]

Since x264 doesn't detect fade-in/fade-out or long pans as a scene change you have to add them manually.Yes, x264 detects fades; that's one of the prime purposes of b-adapt 2.

Also, note that putting I-frames on fades is exactly what you don't want to do. Fades need to be mostly B-frames, and in an ideal encoder, B-frames with weighted P-frame prediction between the P-frames.

[cyberbeing]

Yes, x264 detects fades; that's one of the prime purposes of b-adapt 2.

Also, note that putting I-frames on fades is exactly what you don't want to do. Fades need to be mostly B-frames, and in an ideal encoder, B-frames with weighted P-frame prediction between the P-frames.

Who ever said anything about putting I-frames on fades? You misunderstood me, I'm talking about scene changes not the fades themselves. Once a fade-out/fade-in completes x264 doesn't add an I-frame for the scenechange.

[Dark Shikari]

Who ever said anything about putting I-frames on fades? You misunderstood me, I'm talking about scene changes not the fades themselves. Once a fade-out/fade-in completes x264 doesn't add an I-frame for the scenechange.This is also a bad idea; its a complete waste of bits. Such a high-quality frame should be a P-frame, not an I-frame.

If you mean a higher quality frame to improve the quality of a very static scene after the fade has completed, RDRC (http://pastebin.com/f777a71fc) will do exactly that (in fact, it will do even better than your naive approach; it'll improve quality on a sliding scale towards the end of the fade, so you never see a sudden jump in quality). The macroblock tree patch that failed as a GSOC project would do even better (and without the massive speed hit of RDRC), but unfortunately nobody has volunteered to do the patch since the student bailed, despite it being probably the easiest of the four GSOC projects.

[cyberbeing]

That is what I was talking about. A slowly moving static image fading out while another slowly moving static image is fading in. When I encoded it with r938 last month I had a drop in quality for these areas after the fade until keyint got triggered. When I manually added a I-frame quality improved.

What is RDRC? A new patch? It sounds useful but I don't know how to interpret the diff you posted.

[Dark Shikari]

What is RDRC? A new patch? It sounds useful but I don't know how to interpret the diff you posted.New? No, its very very very old, and very bruteforce. I keep a version updated so as to work on recent versions of x264 (original is from >400 revisions ago).

All it does is for every single frame, encode X future frames depending on that frame for a whole bunch of frame quantizers in order to choose the optimal frame quantizer based on how much a frame's quality affects future frame quality.

[Audionut]

That sounds good. And slow.

[Dark Shikari]

That sounds good. And slow.It isn't as slow as you'd think, because it does the lookahead with ultra-fast encoding settings and skips bitstream coding, too.

If you're already a nut using ref=16, me=tesa, and similar, its not actually much slower.

But yes, its totally impractical, which is why we need MBtree.

[cyberbeing]

Would you be willing to compile a current build with the patch or should I field it for someone else in another thread? Compiling is something I'm not set-up to do.

And from the sound of it, you don't plan to pick up and develop the "macroblock tree" patch and unless someone suddenly volunteers it could stay undeveloped for quite a long time. Correct?

[Audionut]

which is why we need MBtree.

What's the difficulty level and care factor for that.

And also, weighted P-frame prediction.

[Dark Shikari]

And from the sound of it, you don't plan to pick up and develop the "macroblock tree" patch and unless someone suddenly volunteers it could stay undeveloped for quite a long time. Correct?I may do it at some point if I get pissed off enough at it not being done; that's what happened to the project from this summer which I considered to be most important, the B-frame decision project, for which the student literally disappeared on the first week (don't worry, the other two projects were successful, they weren't all this pathetic). I felt this was a bit insulting, as it was my pet project, so to speak. So I spent a week in August and just did the damn thing to get it over with.

I've already received a few volunteers to do the MB tree project, but I haven't actually seen anyone start working on it... :p
What's the difficulty level and care factor for that.

And also, weighted P-frame prediction.MB tree is not that difficult--if you're familiar with graph traversal algorithms it should not be too much of a challenge. Basically its just making a tree of how much "data" each macroblock references from every other and using that tree measure how much each macroblock is referenced in the future, and weight quality based on that number.

Weighted P-frame prediction already has a patch, but there's two catches. First, if I recall correctly, the patch is over 800 revisions old. Second, when it was originally made it didn't help significantly, so clearly it needed a better algorithm.

[gizzin]

I think I ran in to the same problem as this guy maybe a cut a clip, and upload it. I wouldn't consider it a fade in though.

[foxyshadis]

I very seldom encode with what would be considered high quants where blocking can become a major issue, so that is not a problem for me. I have done plenty of tests of my own and I agree with you that x264 does have a good in-loop deblocker, but it still can blur detail if you're not careful even at relatively high bitrates. I'm of a bit different opinion where I usually prefer razor sharp and slightly blocky (no horrid macroblocking please) over soft and slightly blurry. If it becomes too blocky, I aim for lower quants and throw more bitrate at it considering I encode for archiving purposes.

What I don't understand is, why not use a very low deblocking to catch the few blocks that inevitably do slip through? Something like -4:-4 or less. Performance and blurring shouldn't be an issue, because the whole deblocking step will be skipped for almost all blocks anyway. Unless you restrict the encoder to a very low qpmax, it's just there to lessen any visual problems wherever quants jump up, without having to re-encode with a much higher bitrate. It may only save you that trouble on 1 in 3 or 5 of what you now re-encode, would that be worth it?

[Ranguvar]

Desync artifacts don't necessarily look like blocks.

source (http://img140.imageshack.us/img140/7281/sourcefh5.png)
encode with deblock (http://img247.imageshack.us/img247/7414/deblockyt4.png)
encode without deblock (http://img247.imageshack.us/img247/2972/nodeblockhz0.png)
encode with deblock, decode without (http://img213.imageshack.us/img213/6914/desyncxm7.png)

What anime is that, please?
</offtopic>

[Dark Shikari]

What anime is that, please?
</offtopic>Ghost in the Shell 2: Innocence

(I'm also pretty sure that somewhere along the line it crossed the threshold of "this contains too high a percentage of CGI to be an anime anymore".)

[cyberbeing]

What I don't understand is, why not use a very low deblocking to catch the few blocks that inevitably do slip through? Something like -4:-4 or less. Performance and blurring shouldn't be an issue, because the whole deblocking step will be skipped for almost all blocks anyway. Unless you restrict the encoder to a very low qpmax, it's just there to lessen any visual problems wherever quants jump up, without having to re-encode with a much higher bitrate. It may only save you that trouble on 1 in 3 or 5 of what you now re-encode, would that be worth it?

I guess I should have made this clear earlier, I don't usually go below -3:-3 on the in-loop.

[Ranguvar]

The proposal is to use a low value like that WITH decoder deblocking. Even the insane -6:-6 with decoder deblocking will look better than that blockfest of a screenshot.

[Dr.D]

The problem is solved!

Details:
My point was that the codec lowers bitrate too much in fades. For example, size of my encoded test clip was 280K, what is way too small for a good quality. With or without deblocking. Deblocking, with all respect, can't do a magic. Ranguvar rang_x264_r0965-2 bulid has been used for testing.
Then I've downloaded skystrife x264.973.modified build and ... the size magically increased to [/B]730K[/B]! It gave a chance, I played the file and, no surprise, the quality was fine!

Then I've compared the sizes and qualities of different builds. For fair comparison I've used default settings with --crf 25.0:

Official....720.....519,548...bad
Ranguvar.950.....653,958...average
Ranguvar.965.....653,932...average
Ranguvar.965-2..324,079...disaster
Ranguvar.977.....810,358...good

Conclusion (based on 1 sample though, I'm going to make more tests): quality of fades encoding was slightly improved in the last 2 years (except one build what I was "lucky" to get for the initial testing :)), improved a lot in the newest 977 build.

The newest Ranguvar.977 build works fine with the rest of my clip, I'm very impressed, my big thanks to the developers! :thanks:

Question: what can cause the disaster in the Ranguvar.965-2 build and the good improvement in the Ranguvar.977 one?

[Ranguvar]

My 965-2 build used VAQ2mod, among many other odd patches. Output would be drastically different. Please read the notes I post :) Default settings are not default across builds/revisions.
As for the recent build, CRF was adjusted to give more bitrate at a given CRF, due to offset caused by AQ.
Glad to see you sorted the problem!

[Dark Shikari]

Official....720.....519,548...bad
Ranguvar.950.....653,958...average
Ranguvar.965.....653,932...average
Ranguvar.965-2..324,079...disaster
Ranguvar.977.....810,358...goodSurprise surprise, totally useless test is totally useless! :rolleyes:

If your eyes exactly follow the bitrate curve, that is "high bitrate looks better than low bitrate," your test says absolutely nothing because we already know that.

Always do comparisons at the same bitrate, anything else is dishonest and pointless.