Fade-in ugly blocking problem, any progress?

[Dr.D]

Couple of years ago I've tried to move from XviD to x264. Finally I didn't. One of the reasons - subj. Now I've tried again - same thing. The forum search shows that I'm not alone, somebody politely calls that "x264 is not very good with fade-ins", but for me it's just unacceptable, blocking is so-o-o-o ugly that ruins whole video.
This is doesn't make any sense for me - why it's so complicated and hard to fix? In the Wiki H264 article I see:

Weighted prediction, allowing an encoder to specify the use of a scaling and offset when performing motion compensation, and providing a significant benefit in performance in special cases—such as fade-to-black, fade-in, and cross-fade transitions.

as the ADVANTAGE of the H264. I understand that in this areas codec increases quantizer to decrease bitrate. So, for hell sake, just do NOT do that!

Anyway, I'm looking for suggestions to minimize this problem, at least make the blockiness not so ugly. Or I have to don't use x264 for videos with fade-ins?

[Quark.Fusion]

What decoder you use?

[LoRd_MuldeR]

AFAIK the new "B-Frame Decision" patch should allow x264 to handle fades MUCH better ...

[Dark Shikari]

Couple of years ago I've tried to move from XviD to x264. Finally I didn't. One of the reasons - subj. Now I've tried again - same thing. The forum search shows that I'm not alone, somebody politely calls that "x264 is not very good with fade-ins", but for me it's just unacceptable, blocking is so-o-o-o ugly that ruins whole video.
This is doesn't make any sense for me - why it's so complicated and hard to fix? In the Wiki H264 article I see:

Weighted prediction, allowing an encoder to specify the use of a scaling and offset when performing motion compensation, and providing a significant benefit in performance in special cases—such as fade-to-black, fade-in, and cross-fade transitions.

as the ADVANTAGE of the H264. I understand that in this areas codec increases quantizer to decrease bitrate. So, for hell sake, just do NOT do that!

Anyway, I'm looking for suggestions to minimize this problem, at least make the blockiness not so ugly. Or I have to don't use x264 for videos with fade-ins?Posting about a specific problem relating to x264 and fade-ins is fine--making general statements without posting streams of the problem is totally meaningless and makes it impossible to help you or improve the code to fix a specific problem.
I understand that in this areas codec increases quantizer to decrease bitrate. So, for hell sake, just do NOT do that!You seem to understand wrong. Xvid is far more likely to break on fades than x264 is, given the combination of a few factors:

1. No adaptive quantization (unless you're using a VAQ patched version) so the darkest frames of a fade will look awful.
2. The same B-frame decision as x264 (to some extent). x264's improved B-frame decision will be patched into the official version within a few days, which will perform even better on fades.
3. No weighted prediction.
4. No deblocking filter.

If you have found a specific case where x264 has problems with fades, post settings and an output stream. Whining makes it less likely that you will be helped.

[foxyshadis]

as the ADVANTAGE of the H264. I understand that in this areas codec increases quantizer to decrease bitrate. So, for hell sake, just do NOT do that!

x264 has an option to turn this on or off already, it's --weightb, and it's even off by default! But you plainly don't understand it, so I wouldn't do that, as it can only increase quality. It doesn't increase quantizer, it just reduces the residual that has to be encoded, like nearly every option that increases quality. Somehow I'm sure that it's more likely your encoding settings are bad, or even that you've completely turned in-loop deblocking off, use 1-pass ABR, or use too low a bitrate in general.

[Manao]

foxyshadis : not quite : weightb != explicit weighted prediction. Though weightb improves fades, it also improves everything else (except anime), and is actually not explicitely meant to handle fades. Explicit weighted prediction is the tool for that.

[Dr.D]

Posting about a specific problem relating to x264 and fade-ins is fine--making general statements without posting streams of the problem is totally meaningless and makes it impossible to help you or improve the code to fix a specific problem.
Oh-Oh. Your post:
http://forum.doom9.org/showpost.php?p=1039573&postcount=107
"That video seems to be a very bad example overall, for two reasons:
1. Tons of fades, which x264's frame decision algorithm is horrible with."

So, the problem exists, you know about the problem, it's not stream specific and well-known. How my stream can help? For the common well-known problem should be common ways for solution.

But I'll post the stream and screenshots soon.

[Dr.D]

What decoder you use?
CoreAVC.

[Dr.D]

AFAIK the new "B-Frame Decision" patch should allow x264 to handle fades MUCH better ...
I use this build:
http://forum.doom9.org/showpost.php?p=1181245&postcount=1008
The new B-frame decision in.

[Sharktooth]

you should set --b-adapt 2 then

[Sagekilla]

Can you post us a sample that's proving to be troublesome, and settings you used for x264? We can't really help your situation if all you're doing is telling us "it looks bad"

[Dr.D]

Ok, here are the samples:
Original video (http://www.mediafire.com/?hzllgl43rl1)

Default settings (http://www.mediafire.com/?en72ddewwio)
x264.exe --progress --fps 29.97 --crf 25.0 --output "Test Fades Dflt.mkv" "Main.avs" 1280x720

Main settings (http://www.mediafire.com/?vyhq8dx62mn)
x264.exe --progress --threads auto --no-fast-pskip --no-dct-decimate ^
--8x8dct --me umh --subme 7 --bime --level 4.1 ^
--mixed-refs --ref 4 --bframes 3 ^
--b-rdo --psy-rd 1.0:1.0 --trellis 2 ^
--fps 29.97 --aq-strength 1.0 --crf 25.0 ^
--no-psnr --no-ssim ^
--output "Test Fades.mkv" "Main.avs" 1280x720

Screenshot:
http://img255.imageshack.us/img255/4452/testfadesdflt0000380733ij0.th.png (http://img255.imageshack.us/my.php?image=testfadesdflt0000380733ij0.png)

In the movie it looks worse - dancing squares are awful.

[Dr.D]

you should set --b-adapt 2 then
Thank you, but doesn't help too much.

See the problem, x264 decreases bitrate a lot in fades, thanks, it's "smart", but I don't need that! :)

Just a minute, I'm going to try this setting in combination with:
--qpmax 35 --qpstep 20

I've tried this before, helped a bit, but without --b-adapt 2.

[Gabriel_Bouvigne]

It's strange: it looks like your deblocking is off

[Dr.D]

Another "clue":
http://forum.doom9.org/showthread.php?p=1062149#post1062149

"...And Mainconcept/Elecard SDK AVC is by far better than x264 for that:
- Fade scene with wpred for pframe..."

So, 1) the problem exists, it's not a my specific problem, it's by the x264 design; 2) the problem can be fixed (Mainconcept did that).
Why (almost) everybody, including developers, says opposite?

I can understand answers like:
"oh, yes, we know about the problem, but:
- it so hard to fix
- it's just a minor problem, we even don't care to fix that"

but anwsers like - "hmm... fades problem... strange, never heard, fades are fine in x264, it must be your stream-settings-decoder-etc. problem" - are really weird for me.

[Dr.D]

It's strange: it looks like your deblocking is off

If you mean encoder settings - I've tried both, on and off.

If you mean decoder settings -sure it is, because:
1) deblocking blurs.
2) deblocking uses CPU a lot.
3) I need movie what looks good everywhere, on any software or hardware player. I don't like to have a following dialog with everybody whom I'll give a movie - "don't forget to turn on deblocking - what the hell is that?".

I know I shouldn't tell this here :). Using deblocking is super-hyper-mandatory on this forum :).
I'm not going to discuss about that now, may be I'll open a separate topic. I'm just saying - x264 is great codec, I'm completely satisfied of my result with deblocking off, way better than XviD. EXCEPT fading spots. Because x264 lower bitrate too much in this spots, I don't need this.

[Comatose]

3) I need movie what looks good everywhere, on any software or hardware player. I don't like to have a following dialog with everybody whom I'll give a movie - "don't forget to turn on deblocking - what the hell is that?".
Deblocking is on by default...

[Manao]

Dr. D : you're confusing the issue on deblocking. Let's separate inloop-deblocking, which, if used, must be used on both encoder and decoder, and deblocking (as a postprocessing), which is used only on the decoder.

I do hope that in the screenshot you posted, inloop deblocking was on.

[Dr.D]

Dr. D : you're confusing the issue on deblocking. Let's separate inloop-deblocking, which, if used, must be used on both encoder and decoder, and deblocking (as a postprocessing), which is used only on the decoder.

I do hope that in the screenshot you posted, inloop deblocking was on.
Ok, inloop-deblocking is on on both encoder and decoder (I guess, I don't see an option on the CoreAVC decoder);
Decoder's Deblocking oprion is off.

[Sharktooth]

WTF, inloop deblocking should be NEVER disabled! so here's your problem...

[Dr.D]

Deblocking is on by default...
Not neccessary, on ffdshow postprocessing is off.

[fields_g]

Not neccessary, on ffdshow postprocessing is off.

I would venture to guess that most people here don't use post processing deblocking here. When you see deblocking mentioned, it is safe to assume they are talking about inloop deblocking.

[Dr.D]

WTF, inloop deblocking should be NEVER disabled! so here's your problem...
Oh, again, "your problem"...
Please read the thread - it's not a my problem.
Ok, please make your encoding with crf 25 and post the settings and result.

BTW, you can see my settings, inloop deblocking is not disabled.

[nm]

Ok, inloop-deblocking is on on both encoder and decoder (I guess, I don't see an option on the CoreAVC decoder);
Decoder's Deblocking oprion is off.
What do you mean by "Decoder", CoreAVC? If so, you have turned inloop deblocking off in the decoder. The screenshot pretty much confirms this too since such corruption does not happen otherwise. Switch deblocking on and see if it helps.

[Terranigma]

please make your encoding with crf 25 and post the settings and result.


Here's (http://www.mediafire.com/?j4nw08dcmwu) my go at it with a modified revision (968) by TheRyuu.

I've used a cqm, because personally, I always felt that the flat matrix didn't handle fades well, even with psy-rdo and aq.

some notable settings used were:
--nf
--crf 20 (25? wth :eek:)

Conclusion: For fades, Explicit weighted prediction or a cqm would be the way to go. :devil:

[Comatose]

Yeah, 25 is really high :/

[Ranguvar]

The screenshot does indeed look very much like inloop is disabled. Considering inloop is H.264's biggest advantage over Xvid, except perhaps CABAC...

I too prefer slight blocking over blurring. If you want a more "Xvid-esque" feel to your encodes, play with the deblock settings during encode. A good guide is here (http://forum.doom9.org/showthread.php?p=810932#post810932). But turning off deblocking is suicidal. Almost all players support deblocking (pretty much everything except the original Xbox with XBMC, and maybe some PDAs/smartphones... iPod *might * work, PSP works, Archos works, 360 and PS3 work, Blu-ray/HD-DVD work, Popcorn Hour works, etc.). Plus, you make it seem like it's hard to use the inloop during playback? ALL decoders I know off use inloop by default.

And one last thing, inloop does not take a ton of power at all... try and see :)

Sorry if we seem indignant, but you appear to likely be using insane settings and then are accusing x264 of bugs? :) At least hold off on that until it's pretty obvious nothing's going wrong. I'll play your encoded video with deblocking *for sure* on when I get back home and see if I get better results than you.

--b-adapt 2 and --weightb should solve all but the most resistant fade problems without the need for Seppuku-by-CQM ;)

[Gabriel_Bouvigne]

*Yes, x264 is not very efficient when encoding fades. Explicit weighted prediction is not implemented, so it tries to do as best as possible with the other tools (including implicit wpred on B frames), and it could possibly do a better job using explicit wpreds.

*Yes, there seems to be a problem within your settings: you are encoding with deblocking enabled within the encoder, but you manually disabled it within the decoder ? You should make your choice: either you don't touch default deblocking settings (which is enabled by default, for a good reason), or you disable deblocking within the encoder (which will flag the stream as not using deblocking), but you should NOT have deblocking enabled within the encoder and manually disable it within the decoder.

[Dark Shikari]

Looks like a simple case of deblocking off during decoding blamed on x264. I haven't seen one of these in a while :p

[Manao]

Note to all : his commandline doesn't disable deblocking. I raised the issue because his vocabulary was confusing, but I think it did enable inloop deblocking, on both sides

[nm]

Note to all : his commandline doesn't disable deblocking. I raised the issue because his vocabulary was confusing, but I think it did enable inloop deblocking, on both sides
Yes, but it seems that he has turned inloop deblocking off in CoreAVC settings.

[Manao]

He did say :"Ok, inloop-deblocking is on on both encoder and decoder (I guess, I don't see an option on the CoreAVC decoder);"
But then, I (or he, or you, or us) may be confused...

[Quark.Fusion]

CoreAVC.

…the fade in/out bug we fixed as in the new internal version I am using does not have the bug shown.
Maybe it related? Try another decoder.

[Dark Shikari]

He did say :
But then, I (or he, or you, or us) may be confused...You can tell quite clearly from the image that inloop deblocking is off.

Also, as I've said before, its not a bug in CoreAVC because the bug in CoreAVC is only triggered by weighted P-frame prediction, which x264 doesn't use yet.

[Dr.D]

Here's (http://www.mediafire.com/?j4nw08dcmwu) my go at it with a modified revision (968) by TheRyuu.

I've used a cqm, because personally, I always felt that the flat matrix didn't handle fades well, even with psy-rdo and aq.

some notable settings used were:
--nf
--crf 20 (25? wth :eek:)

Conclusion: For fades, Explicit weighted prediction or a cqm would be the way to go. :devil:
Thanks!
This encoding looks fine for me (even with deblocking off on the decoder).
Could you please do the same, but with crf 25? The rest of the movie (without fades) looks fine for me with crf 25, so I don't need to double-size the file only for a few fading spots.

P.S. Just made encoding with my settings, but with --crf 20 - fine too. But, again, I don't need a huge file size increase to fix 1% of movie with fades.

P.P.S. But your encode looks promising - file size is 1.5MB (mine only 500KB). It's exactly what I'm looking for - don't do this stupid "smart" dropping of bitrate in fades!
Could please post or PM your settings and the cqm?

[Audionut]

You can tell quite clearly from the image that inloop deblocking is off.

http://img511.imageshack.us/img511/6295/testfadesdflt0000380733kx4.jpg


An encode at crf 25 with inloop deblocking disabled.
And you wonder why it's a blockfest.

Regardless of what you think the problem might be, I find it highly disrespectful that you tell the devs they should be adjusting their baby because you decide you want it this way or another.

How bout doing the coding yourself, then you can have the encoder doing whatever you want.

[Sharktooth]

again.. dont turn off the inloop deblocking... neither in the encoder (you didnt) or the decoder (you did).
here's again YOUR problem...

[Manao]

Dr. D : please, can you confirm (using a screenshot of CoreAvc or FFdshow's panel configuration) that you didn't force the decoder to disable inloop deblocking. The issue has been so much confused that nobody will believe you if you don't, which would be a shame since if inloop is enable, then you remark is perfectly valid.

[Dr.D]

I would venture to guess that most people here don't use post processing deblocking here. When you see deblocking mentioned, it is safe to assume they are talking about inloop deblocking.
Yes, you are right, inloop deblocking is on by default on ffdshow decoder.

[cyberbeing]

One thing that I have noticed that hasn't been touched on is that the higher strength/threshold you set deblocking the encoder, the more often x264 purposefully blocks in large areas (usually flat) with the expectation of it being deblocked. With a low deblocking setting (say -3,-3) x264 usually seems to try to block in smaller, less obvious areas, likely in attempt to keep PSNR up and maintain perceptive quality. This may or may not be the case in the sample here considering that with a major lack of bitrate there is only so much x264 can do.

To sum it up, at least on anime content, if you are disabling in-loop deblocking in the decoder (for whatever reason) using a low deblock setting or even -no-deblock will usually give you a perceptually less-blocky output at the same bitrate in my experience when doing a multi-pass encode.

Only thing you may want to try is sticking a I frame right before a fade-out, during the 50/50 point of a fade-out/fade-in, and another right after the fade-in is complete with a qpfile and hacked stats (I really wish there was a way to keep b-adapt and scenecut while using a qpfile without having to hack your stats file). At least on anime content, even with scenecut maxed out x264 will forgot to place an I frame after a fade-in/fade-out and just wait for keyint to be triggered which usually results in a lack of quality in those areas. (Off-topic: The same thing also happens with large panning scenes and can cause a huge very abrupt quality jump right in the middle of pan from keyint being triggered no matter what the deblocking setting is) You could also use zones to increase the bitrate in those areas until the problem is solved.

[Manao]

the higher strength/threshold you set deblocking the encoder, the more often x264 purposefully blocks in large areas (usually flat) with the expectation of it being deblocked.You give too much credit to the encoder. x264 ignores deblocking when it decides the type/motion vector of a macroblock. What you see might instead be a side effect of having a deblocked reference frame. Since deblocking improve the reference frame, it might increases the likeliness for skipping the macroblock which in turn might give the impression of larger blocked area (while the edge of that area would not be blocky, the area itself would be squareish).

[cyberbeing]

...What you see might instead be a side effect of having a deblocked reference frame...

I don't see myself giving x264 too much credit because that is exactly what I expect is the reason for that happening. It still doesn't change that x264 behaves in this fashion.

[kemuri-_9]

Only thing you may want to try is sticking a I frame right before a fade-out, during the 50/50 point of a fade-out/fade-in, and another right after the fade-in is complete with a qpfile and hacked stats (I really wish there was a way to keep b-adapt and scenecut while using a qpfile without having to hack your stats file).

true --qpfile disables scenecut and b-adapt, but it has no qualms about you re-enabling it after you call --qpfile
i.e.
--qpfile x.txt --scenecut 50 --b-adapt 1
x264 accepts that and chugs away with a scenecut of 50 and --b-adapt of 1.

granted this should be used at a large risk with the ever increasing specificity of the qpfile:
just specifying a few frames as IDR has only a little risk.
more forced frames -> more risk

[cyberbeing]

true --qpfile disables scenecut and b-adapt, but it has no qualms about you re-enabling it after Ah, I usually stuck qpfile at the end of the command line, I didn't know you had to stick it before scenecut and b-adapt... Thanks for the heads up kemuri.

And yes, I usually only add a few IDR frames in problem areas like fade-in/fade-outs and long pans where x264 starts producing blocky/distorted P and B frames from lack of a good reference frame. I know that excessive use of a qpfile will degrade quality.

[Dr.D]

Ok, here are the samples:
Original video (http://www.mediafire.com/?hzllgl43rl1)

Default settings (http://www.mediafire.com/?en72ddewwio)
x264.exe --progress --fps 29.97 --crf 25.0 --output "Test Fades Dflt.mkv" "Main.avs" 1280x720

Main settings (http://www.mediafire.com/?vyhq8dx62mn)
x264.exe --progress --threads auto --no-fast-pskip --no-dct-decimate ^
--8x8dct --me umh --subme 7 --bime --level 4.1 ^
--mixed-refs --ref 4 --bframes 3 ^
--b-rdo --psy-rd 1.0:1.0 --trellis 2 ^
--fps 29.97 --aq-strength 1.0 --crf 25.0 ^
--no-psnr --no-ssim ^
--output "Test Fades.mkv" "Main.avs" 1280x720

Screenshot:
http://img255.imageshack.us/img255/4452/testfadesdflt0000380733ij0.th.png (http://img255.imageshack.us/my.php?image=testfadesdflt0000380733ij0.png)

In the movie it looks worse - dancing squares are awful.
Ok, I think I have to clarify this little mess about inloop deblocking. Actually, I was a bit confused myself.
1. Personally I use inloop deblocking OFF for both encoding and decoding. I know it's very risky to say so here, same as post good steak recipe in vegetarian forum :). Please do not discuss about this here, I think I'll open a separate thread why, after long experiments, I've decided that for my tasks it's better.

2. Encodes above have inloop deblocking ON. Why? Because if not, I'm 100% sure that only advise will be - "you should not ever-never-forever turn inloop deblocking off". Correct? :)

3. Screenshot above is from Default Settings encode and Decoder inloop deblocking OFF. Why? To show that encoder decreases bitrate way too much. So much, that Decoder's inloop deblocking can't handle it properly. Try to download encoded clip and play with Decoder's inloop deblocking ON. You will see very blurred picture with dancing artefacts. Like "good" old VCD encodes. It's understandable, impossible to make a candle from so low bitrate shit.
Sorry for any misleading caused by not providing all the details at first.

Guys, I'm sure you are looking in wrong direction. With or without inloop deblocking, the logic of encoding fade-ins is wrong - it's lower bitrate too much.
See an another proof.
I just made encoding of 2 clips.
First one (http://www.mediafire.com/?hzllgl43rl1) is the original clip, 55 frames.
Second one (http://www.mediafire.com/?cgzlyu916eg) is the frames 25 - 54 from the original clip. So, the dark and biggest part of fade-in is cut off. 30 frames, just bit more than HALF of the original clip.
What do you think about the sizes of encoded clips (don't worry, with inloop deblocking ON)? Second will be smaller then first by? 50%? 20%? 5%? 1%? No, all wrong.
Encoded second clip (what is the about half of the first) is BIGGER by 37%! 392529 against 286845 bytes. And the quality of the second clip, not surprise, is better, because almost no fade there.
http://www.mediafire.com/?qm3yx9bniyu
http://www.mediafire.com/?sbkvnaiqgcz
Again:

Encoded whole clip: 55 frames, 286845 bytes
Encoded part of the clip: 30 frames, 392529 bytes, 37% more.
Deblocking is ON.


Do you smell a rat? I do the big, fat, stinky one. If you don't, I give up...

My settings:
x264.exe --progress --threads auto --no-fast-pskip --no-dct-decimate ^
--8x8dct --me umh --subme 7 --bime --level 4.1 ^
--mixed-refs --ref 4 --bframes 3 ^
--b-rdo --psy-rd 1.0:1.0 --trellis 2 ^
--fps 29.97 --aq-strength 1.8 --crf 25.0 ^
--no-psnr --no-ssim ^
--output "Fade Test Cut.mkv" "Main.avs" 1280x720

[cyberbeing]

2. Encodes above have inloop deblocking ON. Why? Because if not, I'm 100% sure that only advise will be - "you should not ever-never-forever turn inloop deblocking off". Correct? :)
Well as I said in my post, if you or anybody else viewing your encode never plans to use the in-loop deblocker when decoding, you may get better results turning deblocking down or off in the encoder.

[Sharktooth]

--aq-strength 1.8.. way too much. --no-dct-decimate ... useless... --no-fast-pskip... useless... AND clip i too short...
also... again... DONT DISABLE the inloop deblocker.
you cant whine about blocks while disabling the deblocker...

[Dark Shikari]

3. Screenshot above is from Default Settings encode and Decoder inloop deblocking OFF. Why? To show that encoder decreases bitrate way too muchThat's a completely invalid argument. In loop deblocking is just that--in loop. Every frame depends on the output of the previous frame, so even if the bitrate is enormous, gigantic artifacts can accumulate from frame to frame if you disable the inloop deblocker. It has nothing to do with the quality of the encoding.

[Terranigma]

I see that some of you guys are pointing to in-loop being a problem, and that it should always be used. Well, my sample was encoded with --nf, so there's no in-loop (for the sharpest result possible).

With the progression that's been made to x264 over the past year, it's now possible you can configure x264 so that in-loop is practically useless (imho anyways). :p

[Sharktooth]

wrong... completely. inloop is adaptive, so its not a problem at all. also you can change the adaptiveness bias with the strength and threshold options...