Fade-in ugly blocking problem, any progress?

[cyberbeing]

...you have no rights to criticize the encoder if you dont even know the builds you tested have experimental stuff and some of it has not been made by the x264 devs.

You are making it sound like it's utterly impossible for x264 to have a problem that is not caused by experimental patches.

Dr D. can you please just make some encodes with vanilla builds so experimental patches can be ruled out or confirmed as the cause? This thread is really going around in circles and unless it can be proven one way or another that some vanilla x264 builds have an issue with CRF it's really pointless in debating it any further.

[Sharktooth]

noob number 2... FIRST use the vanilla builds (with correct settings), THEN talk...
this thread is 8 pages of useless discussion since modified builds and unoptimal settings have been used to do the comparisons and tests...

[cyberbeing]

noob number 2... FIRST use the vanilla builds, THEN talk...
this thread is 8 pages of useless discussion since modified builds and unoptimal settings have been used to do the comparisons and tests...

noob number 3... If you read my whole post I just said that.

[Sharktooth]

right... i dont know what i read. i interpreted your post in a completely different way.

[Dr.D]

Sharktooth, your behavior becames funny.

Here is my original question:

Ok, simple question for you.
Let say I have 2 builds, build1 and build2 and encode same clip with crf 25. The clip is 30 min long and has 1 min faded part.
Build1 gives bitrate 200 and awful quality for the faded part and bitrate 2000 for the rest.
Build2 gives bitrate 900 and good quality for the faded part and bitrate 2000 for the rest.
But Build1 encodes faded part better with 2-pass bitrate 900 encoding (as you suggest to test).

Which build handles fades better?

You didn't answer, instead you wrote:

such a build doesnt exist... unless you completely screw x264 by adding experimental patches...

First of all, it shows that you are not in good relationship with logic (I guess it's obvious for everybody, but I can provide an explanation personally for you, if you need).

Second, "this build" (more exactly, a pair of builds) exists. I gave the example for you:

It does.
Not exact numbers, of course, but the same idea:

build.....faded piece..."normal" piece...ratio
Off 720......519,548....5,437,664.......10.5
Rang965-2..324,079....4,299,049.......13.2
Rang977.....822,764....4,106,181........5.0

Even if we don't count Rang965-2 (as "completely screwed x264 by adding experimental patches"), the pair is Off 720 and Rang977.

So, you said a pair with described behavior doesn't exist, I gave you the pair. So, you are wrong. Plain and simple.

Instead of confirming that, you wrote:

numbers tell me you're a noob and as such you should not play with experimental builds that do not represent the x264 true encoding efficiency and quality potential.
that said you have no rights to criticize the encoder and jump to wrong conclusions if you dont even know the builds you tested have experimental patches and some of them have not even been wrote by the x264 devs.
Well, seems to be you finally realized that you are wrong, became nervous (it's the sign) and used a last argument in a discussion, when no another left - blame somebody as a noob.

BTW, aboot noobs. Let say I'm a noob. Nothing wrong with that, I can learn. You - can't, what is much worse. It's impossible to learn without ability to admit own mistakes.

[Dr.D]

You are making it sound like it's utterly impossible for x264 to have a problem that is not caused by experimental patches.

Dr D. can you please just make some encodes with vanilla builds so experimental patches can be ruled out or confirmed as the cause? This thread is really going around in circles and unless it can be proven one way or another that some vanilla x264 builds have an issue with CRF it's really pointless in debating it any further.
See my post http://forum.doom9.org/showthread.php?p=1185024#post1185024.
Can I call 720 build from http://mirror01.x264.nl/x264 as vanilla build? It has the issue.

What exactly are you doubt about? That x264 was not good with fades? If you don't believe me (actually you don't need to believe, you can know if you read my posts and use logic), believe the developer:
http://forum.doom9.org/showpost.php?p=1039573&postcount=107

[Dark Shikari]

See my post http://forum.doom9.org/showthread.php?p=1185024#post1185024.
Can I call 720 build from http://mirror01.x264.nl/x264 as vanilla build? It has the issue.

What exactly are you doubt about? That x264 was not good with fades? If you don't believe me (actually you don't need to believe, you can know if you read my posts and use logic), believe the developer:
http://forum.doom9.org/showpost.php?p=1039573&postcount=107
Its been over a year now, and --b-adapt 2 --weightb will result in correct detection of fades.

Its rather bad form to quote posts from so long ago, especially since:

1. x264's development is ongoing
2. When I said that, I wasn't an x264 developer ;)

[cyberbeing]

Dr. D, I understand what your trying to say from a logical point of view but you're missing something. What Sharktooth is trying to say is that unless all outside variables are eliminated you can't be drawing conclusions. Even Rang977 has patches applied. Now if the limited patches on that build could cause something like this is up for debate, but that's not the point. The inconsistencies from encoding with "unoptimal settings" as Sharktooth put it could be causing what you're seeing as well.

The only way to truly know is to test with no patches whatsoever with an encoding comparison method described by some people in this thread that can be agreed on as valid.

Solid proof is needed not the theoretical proof we have so far. If you want the x264 devs to fix a problem, you need to prove that there is a problem with methods they approve of. For all you know, if you follow what they say to do you may realize that the problem never existed and you were just looking at it the wrong way. On the other hand there may be a surprise that there is in fact a "bug". So far we can't say one way or another. Everybody fighting with each other isn't going to change that.

[Sharktooth]

Sharktooth, your behavior becames funny.

Here is my original question:



You didn't answer, instead you wrote:



First of all, it shows that you are not in good relationship with logic (I guess it's obvious for everybody, but I can provide an explanation personally for you, if you need).

Second, "this build" (more exactly, a pair of builds) exists. I gave the example for you:



Even if we don't count Rang965-2 (as "completely screwed x264 by adding experimental patches"), the pair is Off 720 and Rang977.

So, you said a pair with described behavior doesn't exist, I gave you the pair. So, you are wrong. Plain and simple.

Instead of confirming that, you wrote:


Well, seems to be you finally realized that you are wrong, became nervous (it's the sign) and used a last argument in a discussion, when no another left - blame somebody as a noob.

BTW, aboot noobs. Let say I'm a noob. Nothing wrong with that, I can learn. You - can't, what is much worse. It's impossible to learn without ability to admit own mistakes.
i wont even comment apart the fact you ignore the advices from develpers and other ppl that tell you how to configure the encoder...
you use modified builds with experimental patches...
you fail to see where you're doing wrong comparisons since ppl told you to compare at the same bitrate...
for what concerns me, i have nothing to add except you can stop your crusade against a non existent problems (caused by your misconfiguration and/or the use of old or "modified" builds).
your conclusions and test methods are completely wrong... so you can either:
1 - use vanilla builds and compare with the correct test methodology
2 - shut up
chose one.

[Ranguvar]

I agree... and my r965-2 build was not "completely screwed", it was just very different from most custom builds (lots of experimental patches, some of which were on by default, such as VAQ2mod). Of course the results will be drastically different. Using the same CLI across builds does NOT guarantee the same settings. Revisions change defaults, as do patches. I make my builds with the expectation that the user will do their homework and check out the patches I used, to see what the changes are. If you aren't willing to do that, use vanilla builds.

Please do your homework next time.

But moving on, I believe I am right in believing what you're trying to do is show that x264 is inefficient with fades, by not using deblocking, as that would "cover up" the problem? This is not _entirely_ unreasonable. However, you must see that between the developers who make x264 and a person who has just started using it, we would believe the former more readily. By all means, if you see a better way to work with fades, submit a patch, or point it out :) But when using sane settings (this even includes low deblocking, like -2:-1, if you like sharpness! But enable deblocking on both ends.), and --b-adapt 2 and --weightb, I see no problem with fades. They are never smudged to me, which would indicate that the deblocker is working overtime, unless you take a very quick fade and grab a frame in the middle, in which case the encoder is right to drop quality there, as it will be unnoticeable.

Use a vanilla build, with the options I mentioned, using encode a slow fade, and show us evidence of the deblocker working overtime (blurry). Make sure to give the source, exact settings used, and another frame with source to show that other frames are not hurt, which means you chose a sane bitrate. Do all this, and I think I can assure you you will be heard.

[cyberbeing]

...unless you take a very quick fade and grab a frame in the middle, in which case the encoder is right to drop quality there, as it will be unnoticeable.

To the x264 devs, is the above statement by Ranguvar true? Is x264 designed to "drop quality" where "unnoticeable" when using CRF?

Please no hostility. I am just trying to fully understand how x264 works so then I don't have to make inaccurate assumptions.

As an abstract follow-up question, is unnoticeable a valid reason to drop quality? What if what was thought to be unnoticeable was noticeable after all?

[Ranguvar]

I'm just saying it might, and if so it'd probably be good! Not that it will! I have no clue! I just said that to explain why there may be a quality loss if you have an extremely quick fade, in case it came up :)

The goal of every encoder, after all, is to reduce filesize by compressing where you don't notice it to give more bitrate where you do notice it.

[Dr.D]

Dr. D, I understand what your trying to say from a logical point of view but you're missing something. What Sharktooth is trying to say is that unless all outside variables are eliminated you can't be drawing conclusions. Even Rang977 has patches applied. Now if the limited patches on that build could cause something like this is up for debate, but that's not the point. The inconsistencies from encoding with "unoptimal settings" as Sharktooth put it could be causing what you're seeing as well.
I afraid I've started to miss you point. Or you misses mine.

See the name of the thread - "Fade-in ugly blocking problem, any progress?". I answered few pages ago and said "the case is closed". The answer is "yes". Good progress in the last builds, except disaster in Rang.965-2, and very good progress in Rang.977.

How is your, probably correct theoretical construction, can help to answer my question?

Which exactly part of my conclusion you disagree?

The only way to truly know is to test with no patches whatsSolid proof is needed not the theoretical proof we have so far. If you want the x264 devs to fix a problem, you need to prove that there is a problem with methods they approve of. For all you know, if you follow what they say to do you may realize that the problem never existed and you were just looking at it the wrong way. On the other hand there may be a surprise that there is in fact a "bug". So far we can't say one way or another. Everybody fighting with each other isn't going to change that.
Again, few pages ago I said "the problem is solved".
So, I don't want the x264 devs to fix a problem - already fixed.
I've stopped any fighting since, I'm only defending.
Situation is crystal clear for me.
If somebody has this problem with fades and follow my advice - he will good chance to win.
If somebody will follow Sharktooth or Dark Shikari advice - test a faded piece with same bitrate - he will good chance to loose. But it will be this person's problem.

[cyberbeing]

@Dr D. That post was more that if the problem could be reproduced on a vanilla build it would be interesting. Were Sharktooh and Dark Shikari really telling you to use the same bitrate on just faded portion instead of the whole clip? I don't even know any longer but if you're testing rate control decisions it should be the same avg bitrate for the entire video not just the faded parts.

I'm just saying it might, and if so it'd probably be good! Not that it will! I have no clue! I just said that to explain why there may be a quality loss if you have an extremely quick fade, in case it came up :)

I'm wasn't trying to target you Ranguvar. I just felt it was a good opportunity to throw the question out there.


The goal of every encoder, after all, is to reduce filesize by compressing where you don't notice it to give more bitrate where you do notice it.
Every person is different though and may notice quality in different unlikely areas. What suits one person may not suit another. On that note, something that may be interesting would be fully tweakable presets for rate-control to better suit the types of frames where you want quality to be and have the ability to give things different weights of importance (dark areas, light areas, medium areas, detailed areas, flat areas, gradients, fades, fast moving scenes, slow moving scenes etc). I think AQ does some of this in a generalized fashion, but a streamlined and specialized weight based rate control based on different frame characteristics would be interesting. It would give a level of control beyond what we have today.

[Ranguvar]

I'm wasn't trying to target you Ranguvar. I just felt it was a good opportunity to throw the question out there.
I know :) I wasn't trying to be defensive.

Every person is different though and may notice quality in different unlikely areas. What suits one person may not suit another. On that note, something that may be interesting would be fully tweakable presets for rate-control to better suit the types of frames where you want quality to be and have the ability to give things different weights of importance (dark areas, light areas, medium areas, detailed areas, flat areas, gradients, fades, fast moving scenes, slow moving scenes etc). I think AQ does some of this in a generalized fashion, but a streamlined and specialized weight based rate control based on different frame characteristics would be interesting. It would give a level of control beyond what we have today.
True, but generalizations can, do, and have to be made, based on the average reception different settings get.

[cyberbeing]

True, but generalizations can, do, and have to be made, based on the average reception different settings get.
Nothing wrong with that, but a fully customizable rate control like I described would still be interesting and have it's uses for excessive setting tweakers in maximizing quality where you want it at a given bitrate.

[akupenguin]

To the x264 devs, is the above statement by Ranguvar true? Is x264 designed to "drop quality" where "unnoticeable" when using CRF?
Yes. Which is perfectly equivalent to trying to add quality where it's most noticeable. AQ, as well as all x264 ratecontrol modes other than CQP, do this.
However, note that ratecontrol has no fade detection. B-adapt=2 helps in fades, but even it doesn't know which frames are fades; it's just an algorithm which is closer to RD-optimal in general. There is no parameter you can tune that's "how much to raise/lower quality in fades".

[cyberbeing]

Yes. Which is perfectly equivalent to trying to add quality where it's most noticeable. AQ, as well as all x264 ratecontrol modes other than CQP, do this.
However, note that ratecontrol has no fade detection. B-adapt=2 helps in fades, but even it doesn't know which frames are fades; it's just an algorithm which is closer to RD-optimal in general. There is no parameter you can tune that's "how much to raise/lower quality in fades".

Thank you for the reply akupenguin. Is adding fade detection for ratecontrol impossible? How feasible would it be to develop a full customizable weight based (sort of like a "debug" mode) rate control like I described here (http://forum.doom9.org/showpost.php?p=1185200&postcount=164)?

[Dr.D]

i wont even comment apart the fact you ignore the advices from develpers and other ppl that tell you how to configure the encoder...
you use modified builds with experimental patches...
you fail to see where you're doing wrong comparisons since ppl told you to compare at the same bitrate...
for what concerns me, i have nothing to add except you can stop your crusade against a non existent problems (caused by your misconfiguration and/or the use of old or "modified" builds).
your conclusions and test methods are completely wrong... so you can either:
1 - use vanilla builds and compare with the correct test methodology
2 - shut up
chose one.
Just few advises.
Don't go deeper to the idiotic situation what you made by yourself for yourself.
Don't act like a sort of winner, you completely loose this part of discussion.
Don't masquerade your unability to comment my posts by talking about something completely different.

I'm still waiting your comment of that:

So, you said a pair (of builds) with described behavior doesn't exist, I gave you the pair. So, you are wrong. Plain and simple.

Try to turn on logic (after you answers it's hard to believe you can do that, but try your best) and realize, that I can choose encoder parameters, test clips, decoder parameters, etc. to disprove your statement.

so you can either:
1 - admit you are wrong
2 - shut up
chose one.

[Sagekilla]

Guys, let's cool down a bit please. I don't think we want anyone getting struck for a rule or having this thread closed for whatever reason.

@Dr. D: I think what Sharktooth is trying to say is that you should try to really narrow any variables down to make sure you're getting the complete apples to apples comparison. The more variation (differing bitrate, for example) you have in different aspects, the less reliable your results will be.

[Dr.D]

Dark Shikari, i'm still waiting the answer on my question:

You cannot compare two encodes at different bitrates. End of story.

Ok, simple question for you.
Let say I have 2 builds, build1 and build2 and encode same clip with crf 25. The clip is 30 min long and has 1 min faded part.
Build1 gives bitrate 200 and awful quality for the faded part and bitrate 2000 for the rest.
Build2 gives bitrate 900 and good quality for the faded part and bitrate 2000 for the rest.
But Build1 encodes faded part better with 2-pass bitrate 900 encoding (as you suggest to test).

Which build handles fades better?

Hope you are not so bad with logic as Sharktooth and will not answer "this bulids don't exist" :).

[Sagekilla]

Dr. D, what you describe is more or less a pathological case. Unless you know of a source (and I'm sure all of us would love to see such a source, since it would help improve what's wrong with x264) that exhibits such a horrific variation between two builds, this is more or less a absolute-worst-case-one-in-a-million scenario.

[Dark Shikari]

Dark Shikari, i'm still waiting the answer on my question:

Ok, simple question for you.
Let say I have 2 builds, build1 and build2 and encode same clip with crf 25. The clip is 30 min long and has 1 min faded part.
Build1 gives bitrate 200 and awful quality for the faded part and bitrate 2000 for the rest.
Build2 gives bitrate 900 and good quality for the faded part and bitrate 2000 for the rest.
But Build1 encodes faded part better with 2-pass bitrate 900 encoding (as you suggest to test).

Which build handles fades better?

Hope you are not so bad with logic as Sharktooth and will not answer "this bulids don't exist" :).Such a case surely couldn't exist, since CRF and 2pass use the same ratecontrol equation ;)

[Dr.D]

Guys, let's cool down a bit please. I don't think we want anyone getting struck for a rule or having this thread closed for whatever reason.

@Dr. D: I think what Sharktooth is trying to say is that you should try to really narrow any variables down to make sure you're getting the complete apples to apples comparison. The more variation (differing bitrate, for example) you have in different aspects, the less reliable your results will be.
Well, if you follow the Sharktooth's part of the discussion you will see only 2 points:
1. I'm a noob, so I even don't have rights to use non-vanilla builds, use non-deafult parameters, make conclusions and, of course, discuss with The Man From Heaven.
2. Compare at the same bitrate in any situation, always, forever and ever.

About 1 you can read the thread and make an own conclusion.

About 2 - it's wrong.
See my unanswered question to Dark Shikari and try to answer yourself:

Ok, simple question for you.
Let say I have 2 builds, build1 and build2 and encode same clip with crf 25. The clip is 30 min long and has 1 min faded part.
Build1 gives bitrate 200 and awful quality for the faded part and bitrate 2000 for the rest.
Build2 gives bitrate 900 and good quality for the faded part and bitrate 2000 for the rest.
But Build1 encodes faded part better with 2-pass bitrate 900 encoding (as you suggest to test).

Which build handles fades better?

Isn't it obvious for you that comparing faded part at the same bitrate here is plain stupid? Because this comparing says - Build1 is better. What is wrong - Build2 produses just microscopic larger clip with much better faded part.

Now see - it's not an imaginary situation, it's exactly mine situation. And, I guess, it's typical situation for many CRF users.

[Dr.D]

Dr. D, what you describe is more or less a pathological case. Unless you know of a source (and I'm sure all of us would love to see such a source, since it would help improve what's wrong with x264) that exhibits such a horrific variation between two builds, this is more or less a absolute-worst-case-one-in-a-million scenario.
Everybody could see the source a long time ago :) :
http://forum.doom9.org/showthread.php?p=1183889#post1183889
Just an usual fade-in piece.

[Sagekilla]

I have actually never seen your case. I don't change the in loop deblocker settings (it's enabled and at defaults of 0:0 for the encoder, and CoreAVC has the setting enabled) for the record, and I've not once seen a blocking problem as bad as yours.

Edit: Thank you for directing me to the source. I'll see what I can do to help you out with your situation.

[Sagittaire]

IMO you use the new bframe algo for build 1: imply that fade detection are better. build 1 will use massive bframe number for fade but with high default ratio for bframe. That's mean lower bitrate and lower local quality for fade.

1) change the bframe ratio for better quality
2) use zone option if you want better quality in specific part

crf mode is not constant quality level. You can change the bitrate with the same crf for same build (ratio for frame or psy-rdo can change dramaticaly the bitrate).

[Dr.D]

Such a case surely couldn't exist, since CRF and 2pass use the same ratecontrol equation ;)
The statement "couldn't exist" means doesn't exist unconditionally, so I can choose everything - builds, clips, settings, etc. for disprove.
Correct?

[Dr.D]

I have actually never seen your case. I don't change the in loop deblocker settings (it's enabled and at defaults of 0:0 for the encoder, and CoreAVC has the setting enabled) for the record, and I've not once seen a blocking problem as bad as yours.

Edit: Thank you for directing me to the source. I'll see what I can do to help you out with your situation.
Thank you, but I already solved the problem :) :
http://forum.doom9.org/showthread.php?p=1184605#post1184605
The rest of discussion is the attemts to some people to prove that I'm a noob, made everything wrong and have to shut up :).

But, of course, if you will find something interesting, please let me know.

[Dr.D]

I have actually never seen your case. I don't change the in loop deblocker settings (it's enabled and at defaults of 0:0 for the encoder, and CoreAVC has the setting enabled) for the record, and I've not once seen a blocking problem as bad as yours.
Try to encode my sample with your settings, --crf 25 and Ranguvar 965-2 build http://www.rapidspread.com/redirect?link=495874&hash=rqt1.

[Sagekilla]

I'll get to this tomorrow when I have the time. I need to sleep for now.

[Dr.D]

Let say I have 2 builds, ...

Such a case surely couldn't exist, since CRF and 2pass use the same ratecontrol equation ;)
Hm... hope you are just kidding... If yes, just say that and don't read the rest of the post.

Otherwise you have serious problems with logic too.
I clearly see this scene from you childhood:

Teacher. Let say you have 5 apples. Let say you gave 3 apples to Susan. How many apples left with you?
DS. 5 apples.
Teacher. Why?
DS. I will not give any apples to Susan.

Ok, I slightly modified the question:

Let say I have 2 builds, build1 and build2 and encode 2 clips with crf 25. The 1st clip is faded 2 sec long and 2nd is 8 sec non-faded.
Build1 gives bitrate 1400 and awful quality for the 1st and bitrate 4300 for the 2nd.
Build2 gives bitrate 3600 and good quality for the 1st and bitrate 4100 for the 2nd.
But Build1 encodes faded 1st better with 2-pass bitrate 3600 encoding (as you suggest to test).

Which build handles fades better?

[Dark Shikari]

Hm... hope you are just kidding... If yes, just say that and don't read the rest of the post.CRF and unrestricted 2pass use the same ratecontrol equation.

I cannot imagine a case where they could give any significantly different bitrate distribution. If you can give me such a case, I may find it interesting.

However, since it seems that you cannot understand the most basic of statements, I am not going to bother any more with this ridiculous thread.

[Quark.Fusion]

Lets say build1 uses only b-frames on fades and build2 uses only p-frames on fades, --pbratio defaults at 1.3 — which build handles fades better? :) In CRF mode quality and size depends of how many b-frames you have, not counting AQ. I think it's why devs ask for same bitrate.

Now lets say build1 uses experimental psy-optimisations and build2 not experimental — where you should complain? Note that build1 will produce completely diffirent quality and bitrate at same CRF value.
If you jump from 250 revisions old build to current experimental branch and compare both with some other not-so-experimental branch — what point in it? Either compare current official with old official or current official with current experimental.

Because many ppl try to do that irrelevant compares devs shout at you. And when you shout you don't see all picture. Why others should do correct tests to prove that problem exists if there may not be problem at all?
All that can be easily compared is quality-to-bitrate ratio — all other is heavily relevant on settings and source. Get two related builds (see above) and do two two-pass encodes with same settings, then post x264 log from encode, the source used, produced result and frames that shows the difference. If source is to big — cut out problematic GOPs (try Avidemux).

[Sharktooth]

Just few advises.
Don't go deeper to the idiotic situation what you made by yourself for yourself.
Don't act like a sort of winner, you completely loose this part of discussion.
Don't masquerade your unability to comment my posts by talking about something completely different.

I'm still waiting your comment of that:



Try to turn on logic (after you answers it's hard to believe you can do that, but try your best) and realize, that I can choose encoder parameters, test clips, decoder parameters, etc. to disprove your statement.

so you can either:
1 - admit you are wrong
2 - shut up
chose one.
so if it makes you happy, you won... you won the prize of idiot of the year.
so yes, you win and no, im not unable to comment your posts, i just dont want coz i dont like to play with childish ppl, expecially with you, since you seem to not even know what are you talking about.
so, as i already said, i wont comment anymore. enjoy your prize and continue to show it to the rest of the ppl on the forum as you''re already doing.
thank you... drive thru...

[foxyshadis]

Thread's going nowhere but insults and logic games. Please ignore useless hypothetical questions instead of flaming. Closed.

Some commandments for x264 bug reporting, and this applies to most open-source projects in general:

1. Don't ever report any bug in any patched build. Reproduce it in the latest unpatched build, and report that.
2. If you can't, report the bug to the patch's author, not the main developers (unless they're the same, of course), unless you want to be flamed.

The commandments for discussing any x264 bug report here:

1. Verify that they followed #1 first. Don't flame, just ignore, if they didn't.
2. Don't let it get so far off topic! Open a new thread if you really want to discuss related things that aren't bugs.
3. If something is definitely verified broken, or has already been fixed, stop beating a dead horse (other than following a dev's request to help fix or verify it) and just appreciate it.