Recent BUG in x264 (r936 to r969/977) - ONLY x264.nl build

[patrick_]

I just upgraded my x264 build from r936 to r969 and r977

Both r969 and r977 when encoding my .avs after a while the speed increases to 80+ fps and the output is useless.
It doesn't seem to happen always at the same moment.

r936 worked just fine. I know a lot of new things have been introduced in r969, and for some reason I managed to get a partly encode (1st pass ok, 2nd pass wrong, 3rd pass ok but mostly useless because of the error in the 2nd pass).

There 2 problems now:
- when doing a CRF 18 encode, at about 10% (22 min IFO/VOB) speed increases rapidly and output is corrupted
- when bitrate encoding at the 2nd pass I received warnings that I should lower QP-min (I don't remember the rest of the info, but said something about the QP curve). BTW QP-min = default = 10

I don't know if these problems are related.

----
The point of this message is to act as a bug-report. I guess there's a better place to do so, but I don't know where.

============================
updates:
affects x264.nl build, but not
skystrife (http://forum.doom9.org/showthread.php?p=1184403#post1184403)'s build (includes patches: x264_hrd_pulldown.09_interlace.diff & gcc 3.4.5 fprofiled build with -march=pentium2.)

My computer is a X2 3800+ Windows Vista 32-bits

[Ranguvar]

Provide a source, script, exact settings, and build used.

[patrick_]

I've updated the first post. Uploading the source isn't really easy to do, because it's a VOB. I will have to try other sources to see if it happens to them as well.

[Sharktooth]

script and settings? does it happen with every video you try to encode?

[kemuri-_9]

would you mind watching the peak memory (and virtual memory) as it runs if possible?
this might be a case of hitting the 2GB limit

[patrick_]

I'll try to post a simplified script + settings.

memory usage is about 200MB when the bug falls in.

[Ranguvar]

Please no simplification :)

I'm guessing the problem is source, script, etc., because the only difference with skystrife's build is the hrd pulldown patch. Are you using --nal-hrd?

[burfadel]

The memory usage of x264 in the last month? has significantly reduced, or at least thats what I've experienced! I'm encoding something right now with the normal filters I've used before, x264 only shows as using 128mb or ram under the task manager as where before it would be using up to 1gb! Actually even when encoding with a few things open, it only shows as 1.2gb of ram used in total under task manager with Vista x64.

Going by this, I would find it difficult to think that it could be reaching the 2gb limit, something would have to be seriously wrong. Is your computer overclocked, or ram settings tweaked?

[patrick_]

nope. x264 used to use 1GB, skystrife's build is using 500MB right now.
Anyways it would be strange that a new build would affect that much cpu/ram heat.

[Ranguvar]

Again... we need a piece of your source (use DGSplit to cut) that can replicate the error, and an exact command-line and script.

[patrick_]

I didn't have time to upload anything yet, but the problem is here:
SeparateFields()
DeBlock()
Weave()

line 1 and 3 work fine
line 2 works fine
together I get this AviSynth error randomly: CAviStreamSynth: System Exeption - Access Violation at 0x39e643f, reading from 0x69ac0a6
Now it is an avisynth error, but only affects randomly x264. BTW I also happens with skystrife's build, just for somereason the first time it didn't happen.
It's more likely to happen when using --b-adapt 2

If I can find some time later this week I'll post the source.

[Ranguvar]

Sounds like AviSynth is having trouble reading the source. Either corrupt, or being accessed by another process (AV?), etc?

[patrick_]

I just tried another DVD with the same settings, and everything works fine. So something that has changed recently in x264 causes this error (AV seems related to threading, but I didn't read about any changes in threading), but because it's just in this single dvd, I don't really want to waste more time.

Thanks to all of you for trying to help.

[SZGY]

I'm almost certain I bumped into the same issue. The cmdline I used was

--crf 18 --ref 8 --mixed-refs --no-fast-pskip --bframes 16 --b-pyramid --bime --weightb --trellis 1 --analyse all --8x8dct --direct auto --me umh --threads auto --thread-input --progress --no-psnr --no-ssim

The resulting video displays artifacts when played back using libavcodec (ffdshow) BUT it's perfectly fine using CoreAVC. I also noticed that on certain scene changes the bitrate jumps considerably more with these new x264 builds. Perhaps libavcodec cannot cope with the excessive bitrate?

Setting --qpmin 14 gets rid of the issue while setting anything lower than that brings the blocks back (remember, ONLY libavcodec phailz).

This may very well be libavcodec's fault. Needs more thorough testing still.