Nasty blocks on x264 encode

[omion]

I just finished encoding my first HD source in x264, and I noticed a little block in one scene, right next to her nose:
http://omion.dyndns.org/other/x264-blocks-20060110/initial1.jpg
Initial encode [812KiB] (http://omion.dyndns.org/other/x264-blocks-20060110/initial.mkv)
I can see it with both ffdshow and coreAVC decoders, so it's not a decoder issue. I don't see how it could have been the result of a good decision on the part of the encoder, so I tried to isolate it. It seemed to be very specific to the options used: adding --no-asm didn't fix it, but changing most other options did. A list of all the command lines I used, along with their results, is here (http://omion.dyndns.org/other/x264-blocks-20060110/settings.txt).

When I was cropping the source file to a size more reasonable for the internet, I found that if I cropped off the bottom I could get some really hairy blocks:http://omion.dyndns.org/other/x264-blocks-20060110/nasty.jpg
Tweaked encode [559KiB] (http://omion.dyndns.org/other/x264-blocks-20060110/nasty.mkv)
The command line I used was:
x264 --mixed-refs --crf 25 --subme 5 --ref 8 --progress --me umh --merange 16 --bframes 3 --weightb --b-pyramid --analyse all -o "Error.mkv" ".\error.avs"

I tried this with both r396 and 397, and they both do the same thing.

I have a clip of the cropped source, as well as an AVIsynth script, for others to test out here (http://omion.dyndns.org/other/x264-blocks-20060110/source.7z). [NOTE: This is a 26MiB file, so if you're not going to try to fix it don't download it. I host all this stuff off my home computer, and I don't want to melt my DSL.]

It looks similar to the old "flat blue blocks" problem, except that where it occurs isn't flat or blue... Also, I just got done with 18 anime episodes, and I didn't have the problem there at all. The only thing I did differently this time was encoding 1600x880 instead of 720x480.

Any ideas?

[kotrtim]

--no-fast-pskip
tried this?

Fast P-frame skip
Enable Only when experiencing block

quoted from megui x264

[bob0r]

If --no-fast-pskip solves his problem its about time its enabled by default.

I am downloading the source file and the .mkv file, ill mirror it on http://files.x264.nl/omion

[akupenguin]

Using the command written in that avs, I can't reproduce it.
http://students.washington.edu/lorenm/src/x264/ok.mkv

[Sharktooth]

@omion: Uhm, it smells like... something on your end (maybe your system) :)

[CruNcher]

first you should really tell us wich build did you used as we have 2 binaries floating around allready.

[omion]

first you should really tell us wich build did you used as we have 2 binaries floating around allready.
I used Sharktooth's builds 396 and 397. They both did it. I'll test out some other builds.

I did multiple encodes with the same settings, and they are bit-identical, so it's not a hardware bug (I'm not overclocking)

--no-fast-pskip does fix it, but so does "--subme 6", "--me hex", and "--crf 24". So I don't know what actually helps, and what changes the scene to not produce the problem. I figured --no-fast-pskip was for minor blocking, not the flagrant ones that I'm getting.

I'll try some other builds (I've got a bunch of old ones) and see if anything fixes it.

edit:
The problem happens on builds 379 and above. Builds 365 and below don't do it. However, I don't know if that's where the problem started, or where the the settings were changed to allow the problem to happen on this scene. Build 365, however, puts up a warning at the beginning:
x264 [warning]: DPB size (16896000) > level limit (12582912)Don't know if it's related or not; no other builds do that.

edit2:
I have the 64-bit version of Windows, but I've never seen that make a difference with non-driver programs

edit3:
I'm also testing it out on my old P2 laptop. It's going REALLY SLOWLY. Don't encode HD content on a P2. :p

[Sharktooth]

379 is the same as 378. And however yes, it could be an OS compatibility issue. WOW is not perfect.
Maybe it doesn't like compiler profiling (IIRC 379 had profiling enabled).

[bob0r]

@omion
Try my compile from http://x264.nl and tell me how it goes.

Sharktooth adds patches that may not be working so well yet.

Pengvado created ok.mkv, probably with x264-SVN.

[omion]

@omion
Try my compile from http://x264.nl and tell me how it goes.

Sharktooth adds patches that may not be working so well yet.

Pengvado created ok.mkv, probably with x264-SVN.
Ah. That did it. I get bit-identical to aku's ok.mkv with your build.
The one thing I don't like about your builds is the installer; I don't need one. Is there any way to download just the executable like Sharktooth's "lite" package?

379 is the same as 378. And however yes, it could be an OS compatibility issue. WOW is not perfect.
Maybe it doesn't like compiler profiling (IIRC 379 had profiling enabled).
I finished the encode with your r398 on my P2 laptop (at a whopping 0.03fps) and it has the same problem. The file is identical to the broken one. I think there might be a problem with a patch somewhere.

Also, I don't have any builds between 365 and 379, so all I know is the problem starts manifesting itself between those two.

[bob0r]

Nop, just the installer, its not so bad to install, copy x264.exe and uninstall if you must. I do the same myself, the installer is clean and easy, so good to use.

Very interesting some patch is causing this, i am sure the developers want to know what is causing this, and since you gave us the source i am sure the problem will be pinpointed, good work!

[CruNcher]

Problem found it's the combination of x264_check_subpel_predictors.1.diff with x264_new_decimation.2.diff that causes this behaviour, still investigated stay tune :)

[Sharktooth]

@Cruncher: interesting, thanks for testing and investigation:)

[Sharktooth]

r399 should now fix the problem.
please test it.

[omion]

r399 should now fix the problem.
please test it.
Nope. Still does the same kind of thing. It looks a bit different now, but still major blocking.
@CruNcher: does 399 work for you?

[Sharktooth]

Can you post the screenshots?

[omion]

Here y'go (398 then 399):
http://omion.dyndns.org/other/x264-blocks-20060110/nasty.jpg http://omion.dyndns.org/other/x264-blocks-20060110/nasty399.jpg
New MKV from 399 [561KiB] (http://omion.dyndns.org/other/x264-blocks-20060110/nasty399.mkv)

[akupenguin]

Does this fix it?
mv_clipping.diff (http://students.washington.edu/lorenm/src/x264/x264_mv_clipping.0.diff)

[bob0r]

@omion

Email me the files to files@x264.nl (jpg/mkv, remember gmail has 20mb limit i believe). Ill reply to your email with the links (http://files.x264.nl/omion/), then you can post them (or edit your previous post(s)).

[bob0r]

@omion

Try x264 revision 405 (split mv_range enforcement from edge-of-frame clipping. fixes an occasional artifact with long mvs. ) should fix your problem, pengvado/akupenguin fixed it! (get SVN from x264.nl, try with extra patches from sharktooth later)

[CruNcher]

@omion
when sharktooth is updateing his build you should have no problems anymore with those settings :)

@sharktooth
you have to update the check subpel predictors patch or it will crash
change

- const int mx = x264_clip3( mvc[i][0], h->mb.mv_min[0], h->mb.mv_max[0] );
- const int my = x264_clip3( mvc[i][1], h->mb.mv_min[1], h->mb.mv_max[1] );
+ const int mx = x264_clip3( mvc[i][0], h->mb.mv_min_spel[0], h->mb.mv_max_spel[0] );
+ const int my = x264_clip3( mvc[i][1], h->mb.mv_min_spel[1], h->mb.mv_max_spel[1] );

Alex_W finally found the issue :)

[Sharktooth]

r406A

[omion]

Sharktooth's 406 shows no artifacts. Thanks, everyone, for figuring this out! :thanks:

[Sharktooth]

406 does not have the check_subpel_predictors patch.
406A has the new check_subpel_predictors patch.
Try both and post your findings.

[omion]

Oops. That test was on 406A. Where can I find the normal 406?

[Sharktooth]

if 406A works then there's nothing more to test :D

[omion]

Sounds good. Thanks again!