Heavy x264 Artifacting Recently Observed in Specific Conditions

[CarlEdman]

I have been using the x264 CLI to archive my DVD library and newly bought DVDs for some years now and am generally between happy and ecstatic with the results. However, on a recent batch of files I observed some artifacting I've never seen with x264 before.

The common features are:
(*) Heavy blocking/pixelation extending over changing parts of several frames.

(*) Occurs always right after a scene-cut while the new scene contains heavy movement from the beginning.

(*) May disproportionately affect small clips. I have only seen it in half a dozen short (3-10 minute) videos, not in any full length movie.

(*) May be more common in CGI/animation than in filmed material, though I have observed it in both.

The version:

% x264 --version
x264 0.114.1924 08d04a4
(libswscale 0.12.0)
(libavformat 52.103.0)
(ffmpegsource 2.15.0.1)
built on Mar 25 2011, gcc: 4.6.0 20110322 (prerelease)
configuration: --bit-depth=8
x264 license: GPL version 2 or later
libswscale/libavformat/ffmpegsource license: GPL version 3 or later

It is the x64 version running on Windows 7, if that is not clear from the above.

The command line:

avs2yuv -o - "<name>.video.avs" | x264 --tune animation --preset veryslow --crf 18.0 --sar 8:9 --profile high --level 3.0 --fps 30000/1001 --output "<name>.264" --demuxer y4m -

The avisynth file:

SetMTMode(2,0)
DGDecode_mpeg2source("<name>.d2v", info=3, idct=3, cpu=6)
ColorMatrix(d2v="<name>.d2v", interlaced=true)
TomsMoComp(1,5,1)
autocrop(threshold=30,wMultOf=4, hMultOf=2,samples=51, mode=0)
Interleaved2Planar()
super = MSuper(planar=true)
bv1 = MAnalyse(super, isb = true, delta = 1, overlap=4)
fv1 = MAnalyse(super, isb = false, delta = 1, overlap=4)
bv2 = MAnalyse(super, isb = true, delta = 2, overlap=4)
fv2 = MAnalyse(super, isb = false, delta = 2, overlap=4)
bv3 = MAnalyse(super, isb = true, delta = 3, overlap=4)
fv3 = MAnalyse(super, isb = false, delta = 3, overlap=4)
MDegrain3(super,bv1,fv1,bv2,fv2,bv3,fv3,thSAD=400,planar=true)
Planar2Interleaved()
Distributor()

I'd be happy to upload clips demonstrating the phenomenon if (a) somebody can recommend a public server to store files up to 20 MByte in size, and (b) it is not regarded as a violation of forum rules to upload clips from copyrighted materials (for which I own the DVD).

Any suggestion?

[Dark Shikari]

mediafire.com

Are you sure the bug isn't in your input script? Say, a bug in Avisynth-MT or MVTools? It's rather jumping the gun to claim "heavy x264 artifacting" when x264 is only one of a half-dozen different apps you're using, any of which could be causing your problem.

[Midzuki]

Just out of curiosity,

why are you using avs2yuv?

Was your x264.exe compiled without Avisynth support?

[CarlEdman]

Midzuki, my avisynth filters--while extremely helpful, in my opinion--are very memory hungry. When they need to share an address space with x264, they run out of address space on HD material (even though the machine has plenty of RAM). When they run in separate address spaces (as they do with avs2yuv and the pipes), there is no problem. Also, I seem to recall that 64-bit x264 at some point had some issues with avisynth (which was then and may still be heavily tied to 32-bit architecture thanks to its filters). Using the pipe side-steps the problem (which, for all I know, may have been fixed since then).

[CarlEdman]

Dark Shikari,

it is of course perfectly reasonable to suspect the filters. The reason I did not do so is that I've been using the same filters (both binaries and calls in the script) for more than a year and never observed any such artifacting, while I keep updating to the latest x264 version whenever you release new source.

And it turns out not to be the filters. I've uploaded two copies of the same encode to mediafire, one with the full filters quoted above, one without any filters (expect the unavoidable dgindex input filter). The latter is larger in filesize, but the artifacting looks the same to me in both versions.

However, it also turns out not to be x264. Strangely, it turns out to be mkvtoolnix. The original mkv (generated using MakeMKV from the DVD) against which I compared the final result, did not display any artifacts, so my suspicion fell on x264. But then I examined the intermediate .mpg extracted (using the latest mkvextract) from the .mkv in order to generate a dgindex file and--much to my surprise--the intermediate .mpg contained the same artifacts, long before x264 touched the contents. I am completely baffled how extracting a track from a container format using the official tools can corrupt it, but that is what clearly happened.

My apologies to Dark Shikari for casting unfounded aspersions on his priceless codec and thanks for keeping up the good work (including answering forum questions)!

[Midzuki]

Originally Posted by CarlEdman

Also, I seem to recall that 64-bit x264 at some point had some issues with avisynth (which was then and may still be heavily tied to 32-bit architecture thanks to its filters). Using the pipe side-steps the problem.

My bad. Since I've never used a 64-bit x264.exe, it would be somewhat-"natural" that I forgot about such an "irrelevant" detail.
Gomen nasai

[CarlEdman]

For what it is worth, this is apparently a known, but not-going-to-be-fixed-any-time-soon bug of mkvtoolnix. Original Bug Report; My bug report.

[Midzuki]

Originally posted by Carl Edman :

For what it is worth, MPEG2 tracks extracted with the mkvextract '--raw' option to '.m2v' files do not appear to exhibit the same artifacts as those extracted without '--raw' from the same source MKV file.

Well, considering that AVI-Mux GUI only extracts mpv-in-mkv as "raw" streams,... IIRC, old versions of mkvextract wrapped the "non-raw" outputs in an AVI file — which, in many/most cases, is not a very-good idea, to say the least. :-/

[CarlEdman]

The maintainer of mkvtoolnix appears to have fixed the not-going-to-be-fixed-any-time-soon bug. From the change log of mkvtoolnix 4.7, released today:

2011-04-15 Moritz Bunkus <moritz@bunkus.org>
* mkvextract: bug fix: Fixed extraction of MPEG-1/2 video tracks whose sequence headers change mid-stream but whose key frames are not all prefixed with a sequence header. Fix for bug 556.