x264 cli bugreports

[Doom9]

This thread is for x264.exe bugeports only. VfW users please go here (http://forum.doom9.org/showthread.php?t=108569)

[foxyshadis]

x264 won't output stats in crf mode even if told to. More feature request than bug. I guess I should have checked here before asking for it in Megui!

[ChronoCross]

x264 won't output stats in crf mode even if told to. More feature request than bug. I guess I should have checked here before asking for it in Megui!

what kind of stats are you talking about? Are you talking about Time, fps, bitrate?

[foxyshadis]

Sorry, I meant --stats "x.stats" for use if the crf is unsatisfactory and a second pass needs to be run at a higher bitrate.

[GodofaGap]

Do you have "--pass 1" in your commandline?

[foxyshadis]

No, and it works with it. That's why I was so confused, since I was sure it'd been working last week! Thanks.

[akupenguin]

Well, then you didn't tell it to output stats.
"--stats" just specifies a filename, and is not required (there is a default name). You also need a --pass 1/2/3 to tell it what to do with the statsfile.

[IgorC]

Note of informative character.

There was an AQ patch aplied for revs 466 and higher.
AQ from Sharktooth old build (380-400) wasn't at state of art but somehow brings better quality at (very) low bitrate.
New unofficial AQ patch of 466 doesn't do even half of that.
Some guys are still compiling new rev + AQ patch for test only.

[hpn]

AQ from Sharktooth old build (380-400) wasn't at state of art but somehow brings better quality at (very) low bitrate. New unofficial AQ patch of 466 doesn't do even half of that.

Looking at the source both AQ patches (398 and 466) seem almost identical. Akupenguin just replaced "h->mb.pic.i_stride[x]" with "FENC_STRIDE" (whatever that means) and aligned the new patch offsets with a most recent 466 revision, so at first I thought they were only cosmetic changes. But I just did 2 tests with both patches and got different encodes - the 398 encode was identical to an encode without --aq-strength and --aq-sensitivity at all (a phenomenon already discussed here), while the 466 produced a lower PSRN encode, but with better looking flat backgrounds (to my eyes). I'm still unsure which patch "is better", 398 or 466, or what sets of --aq-strength and --aq-sensitivity options really make difference and in what cases. It's obvious that AQ needs some thorough analysis by the devs in the future, that's why it's not in the SVN, so use AQ for nothing but tests only. I just posted both 398 and 466 AQ builds here (http://mujweb.cz/www/x264/)

[mindphuk]

Hi I already shot this to the x264-devel mailinglist and got an answer there. Unfortunately I could not do much with the answer, because it was technically to special for me.
Maybe here someone can help me:
------------------
Original message:
------------------
I am trying to encode different movies to h264 using ffmpeg and x264.
The system where the encoding happens, is a 64bit Gentoo linux (stage
3).
I tried different parameters of ffmpeg but can not get rid of the grev
fog in black areas of the videos. The problem appears only on some
frames and lasts different times. It seems that it depends on the GOP
setting: When I set GOP to high (300 or more), the problem apears only
some times but always on the same input files and the same time.
ffmpeg shows an output of the library telling something about an
overflow.

I will attach some screenshots of the problem and a logfile.

Greetings
G.K.


GOP size of 18

user ~> ffmpeg -y -i Speedball2_Trailer3.mov -vcodec h264 -b 7000k -bf 3 -g 18 -s 1280x720 -acodec mp3 -ab 96k Speedball2_Trailer3.avi
FFmpeg version SVN-r9361, Copyright (c) 2000-2007 Fabrice Bellard, et al.
configuration: --enable-libxvid --enable-libmp3lame --enable-libfaac --enable-gpl --enable-libx264 --enable-libfaad --enable-shared
libavutil version: 49.4.0
libavcodec version: 51.40.4
libavformat version: 51.12.1
built on Jun 19 2007 10:33:31, gcc: 4.1.1 (Gentoo 4.1.1-r3)
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x2ac7d7afc040]negative ctts, ignoring
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'Speedball2_Trailer3.mov':
Duration: 00:01:53.0, start: 0.000000, bitrate: 8215 kb/s
Stream #0.0(eng): Audio: mpeg4aac, 48000 Hz, stereo
Stream #0.1(eng): Video: h264, yuv420p, 1280x720, 29.01 fps(r)
Output #0, avi, to 'Speedball2_Trailer3.avi':
Stream #0.0: Video: libx264, yuv420p, 1280x720, q=2-31, 7000 kb/s, 29.01 fps(c)
Stream #0.1: Audio: libmp3lame, 48000 Hz, stereo, 96 kb/s
Stream mapping:
Stream #0.1 -> #0.0
Stream #0.0 -> #0.1
[libx264 @ 0x2ac7d7fcb790]using cpu capabilities: MMX MMXEXT SSE SSE2 3DNow!
Press [q] to stop encoding
[libx264 @ 0x2ac7d7fcb790]OVERFLOW levelcode=4377
[libx264 @ 0x2ac7d7fcb790]OVERFLOW levelcode=4377
frame= 10 fps= 0 q=4.0 size= 15kB time=0.2 bitrate= 607.8kbits/s
frame= 20 fps= 20 q=4.0 size= 20kB time=0.6 bitrate= 291.2kbits/s
[libx264 @ 0x2ac7d7fcb790]OVERFLOW levelcode=4377
[libx264 @ 0x2ac7d7fcb790]OVERFLOW levelcode=4377
frame= 27 fps= 17 q=2.0 size= 110kB time=0.8 bitrate=1133.2kbits/s
frame= 32 fps= 16 q=4.0 size= 243kB time=1.0 bitrate=2064.4kbits/s
(...)
frame= 229 fps= 13 q=6.0 size= 7151kB time=7.8 bitrate=7553.6kbits/s
[libx264 @ 0x2ac7d7fcb790]OVERFLOW levelcode=4377
[libx264 @ 0x2ac7d7fcb790]OVERFLOW levelcode=4377
frame= 236 fps= 13 q=4.0 size= 7179kB time=8.0 bitrate=7354.3kbits/s
(...)
frame= 255 fps= 13 q=2.0 size= 7212kB time=8.7 bitrate=6828.8kbits/s
[libx264 @ 0x2ac7d7fcb790]OVERFLOW levelcode=4377
[libx264 @ 0x2ac7d7fcb790]OVERFLOW levelcode=4377
frame= 262 fps= 13 q=4.0 size= 7277kB time=8.9 bitrate=6703.5kbits/s
frame= 269 fps= 13 q=2.0 size= 7402kB time=9.1 bitrate=6638.8kbits/s
[libx264 @ 0x2ac7d7fcb790]OVERFLOW levelcode=4219
[libx264 @ 0x2ac7d7fcb790]OVERFLOW levelcode=4219
frame= 276 fps= 13 q=4.0 size= 7580kB time=9.4 bitrate=6623.1kbits/s
(...)
frame= 289 fps= 13 q=4.0 size= 8072kB time=9.8 bitrate=6731.7kbits/s
[libx264 @ 0x2ac7d7fcb790]OVERFLOW levelcode=4141
[libx264 @ 0x2ac7d7fcb790]OVERFLOW levelcode=4141
frame= 293 fps= 13 q=3.0 size= 8342kB time=10.0 bitrate=6860.5kbits/s
frame= 299 fps= 13 q=6.0 size= 8635kB time=10.2 bitrate=6957.1kbits/s
frame= 304 fps= 13 q=7.0 size= 8928kB time=10.3 bitrate=7073.6kbits/s
(...)
no more errors here...
(...)
frame= 3280 fps= 8 q=-1.0 Lsize= 94184kB time=113.0 bitrate=6826.9kbits/s
video:92660kB audio:1325kB global headers:0kB muxing overhead 0.211337%
[libx264 @ 0x2ac7d7fcb790]slice I:186 Avg QP:20.54 size: 51168
[libx264 @ 0x2ac7d7fcb790]slice P:912 Avg QP:21.22 size: 41287
[libx264 @ 0x2ac7d7fcb790]slice B:2182 Avg QP:23.47 size: 21864
[libx264 @ 0x2ac7d7fcb790]mb I I16..4: 54.4% 0.0% 45.6%
[libx264 @ 0x2ac7d7fcb790]mb P I16..4: 50.0% 0.0% 0.0% P16..4: 26.8% 0.0% 0.0% 0.0% 0.0% skip:23.2%
[libx264 @ 0x2ac7d7fcb790]mb B I16..4: 22.2% 0.0% 0.0% B16..8: 42.0% 0.0% 0.0% direct: 3.5% skip:32.3%
[libx264 @ 0x2ac7d7fcb790]final ratefactor: 19.80
[libx264 @ 0x2ac7d7fcb790]SSIM Mean Y:0.9796557
[libx264 @ 0x2ac7d7fcb790]kb/s:6714.0

Same video with GOP size of 500:

FFmpeg version SVN-r9361, Copyright (c) 2000-2007 Fabrice Bellard, et al.
configuration: --enable-libxvid --enable-libmp3lame --enable-libfaac --enable-gpl --enable-libx264 --enable-libfaad --enable-shared
libavutil version: 49.4.0
libavcodec version: 51.40.4
libavformat version: 51.12.1
built on Jun 19 2007 10:33:31, gcc: 4.1.1 (Gentoo 4.1.1-r3)
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x2b2e462d5040]negative ctts, ignoring
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'Speedball2_Trailer3.mov':
Duration: 00:01:53.0, start: 0.000000, bitrate: 8215 kb/s
Stream #0.0(eng): Audio: mpeg4aac, 48000 Hz, stereo
Stream #0.1(eng): Video: h264, yuv420p, 1280x720, 29.01 fps(r)
Output #0, avi, to 'Speedball2_Trailer3.avi':
Stream #0.0: Video: libx264, yuv420p, 1280x720, q=2-31, 7000 kb/s, 29.01 fps(c)
Stream #0.1: Audio: libmp3lame, 48000 Hz, stereo, 96 kb/s
Stream mapping:
Stream #0.1 -> #0.0
Stream #0.0 -> #0.1
[libx264 @ 0x2b2e467a4790]using cpu capabilities: MMX MMXEXT SSE SSE2 3DNow!
Press [q] to stop encoding
[libx264 @ 0x2b2e467a4790]OVERFLOW levelcode=4377
[libx264 @ 0x2b2e467a4790]OVERFLOW levelcode=4377
frame= 11 fps= 0 q=4.0 size= 16kB time=0.2 bitrate= 531.6kbits/s
(...)
no more errors here
(...)
video:92413kB audio:1325kB global headers:0kB muxing overhead 0.211776%
[libx264 @ 0x2b2e467a4790]slice I:7 Avg QP:20.43 size: 44850
[libx264 @ 0x2b2e467a4790]slice P:824 Avg QP:21.02 size: 44524
[libx264 @ 0x2b2e467a4790]slice B:2449 Avg QP:22.98 size: 23531
[libx264 @ 0x2b2e467a4790]mb I I16..4: 54.0% 0.0% 46.0%
[libx264 @ 0x2b2e467a4790]mb P I16..4: 54.3% 0.0% 0.0% P16..4: 24.3% 0.0% 0.0% 0.0% 0.0% skip:21.4%
[libx264 @ 0x2b2e467a4790]mb B I16..4: 22.7% 0.0% 0.0% B16..8: 42.8% 0.0% 0.0% direct: 3.8% skip:30.8%
[libx264 @ 0x2b2e467a4790]final ratefactor: 19.50
[libx264 @ 0x2b2e467a4790]SSIM Mean Y:0.9798082
[libx264 @ 0x2b2e467a4790]kb/s:6696.5

While GOP below 10 produces a lot of errors...

------
Reply:
------

When encoding CAVLC in Baseline or Main profile, there is a limit on the
size of any one DCT coefficient. This limit can only be reached with very
low QP or with extreme quantization matrices, so it shouldn't normally be
a problem, but ffmpeg has lousy defaults (such as min QP=2) which allow
such extreme cases.
And even though larger values are allowed in High profile (it needs them
to represent higher sample depth), I haven't implemented that in x264.
There is no direct relation between GOP size and overflow, though I-frames
are lower QP than P-frames and I-blocks are more likely to contain large
coefficients than P-blocks.

(pick any one of the following)
What you can do about it without modifying any source code:
Enable CABAC.
Set min QP to something reasonable (like x264's default 10).

What you can do about it with modifying source code:
Add codec-specific defaults in ffmpeg so that h264 doesn't default to
insane settings.
Make x264 encode that coefficient using High profile's larger level_code,
resulting in a syntactically valid stream but violating profile
constraints.
Make x264 re-encode the macroblock with a higher QP.
Make x264 re-decode the macroblock so that the reconstructed picture
matches the bitstream; there will still be an artifact but it will be
localized to a few blocks and fixed in the next frame.

[nm]

What you can do about it without modifying any source code:
Enable CABAC.
Set min QP to something reasonable (like x264's default 10).
You were suggested to use ffmpeg parameters -coder 1 -qmin 10
To get an encode with reasonable quality instead of trash, you should also tweak other parameters. Take a look at Robert Swain's FFmpeg x264 encoding guide (http://rob.opendot.cl/index.php/useful-stuff/ffmpeg-x264-encoding-guide/).

[MasterNobody]

I found that x264 produces different files for my sample file when using next options:
x264.exe -B 800 --8x8dct -A p8x8,i8x8,i4x4 --me umh --subme 6 --trellis 2 --threads 5 --quiet --no-psnr --no-ssim -o output.264 test.aviI think that this bug somehow connected with use of multithreading. I suggest that it is somewhere in encoder\analyse.c x264_mb_analyse_init function but I am not sure. For testing I attached archive with my samle file and cmd-file for automation (it produces 100 files with my options).

[imcold]

It may be because using threads leads to non-deterministic encoding, so it's rather a feature.

[Sagekilla]

It may be because using threads leads to non-deterministic encoding, so it's rather a feature.

Especially with that many threads.. If you're encoding at sub-HD resolutions on a dual core, you'll run into that kind of problem. Unless you actually have more than 2 cores I'd advise against using that many threads.

[check]

No, x264 should still be deterministic with multiple threads. It is only not when you set --non-deterministic.

[MasterNobody]

As check rightly said it must be deterministic indifferently how much threads it use if not using --non-deterministic option. If you add --non-deterministic to my options all files will be different, but without it only some rare files are different (that is why I create 100 files) due to some bug (it is not proposed behavior surely).

[imcold]

I see... sorry, my bad.

[squid_80]

The matroska writer seems to leak small amounts of memory. I made some small changes to matroska.c:static void mk_destroyContexts(mk_Writer *w) {
mk_Context *cur, *next;

for (cur = w->freelist; cur; cur = next) {
next = cur->next;
free(cur->data);
free(cur);
}

for (cur = w->actlist; cur; cur = next) {
next = cur->next;
free(cur->data);
free(cur);
}

for (cur = w->frame; cur; cur = next) {
next = cur->next;
free(cur->data);
free(cur);
}

for (cur = w->root; cur; cur = next) {
next = cur->next;
free(cur->data);
free(cur);
}

w->freelist = w->actlist = w->root = w->frame = NULL;
} which seems to have fixed it. I didn't really spend much time on it though, should probably be checked more closely.

[kemuri-_9]

Hey, I updated to the newest rev of x264 (r886), and i'm coming across some problems when running it after compilation in VC9.

It constantly throws exceptions when trying to run it with asm optimizations.
However, when i turn off asm it runs just fine.
So has there been a regression in the asm code to be incompatible with VC? since the precompiled mingw one on the site still runs fine...

---

in the VC debugger it prints off
movaps xmm0,xmmword ptr [esi+ecx]
for
Unhandled exception at 0x004373a2 in x264.exe: 0xC0000005: Access violation reading location 0x00000000.

---
Running on an AMD Athlon 64 X2 4200+ (Manchester) on XP x64
detected ASM opts (when enabled) are MMX2 and SSE2Slow

[Dark Shikari]

So has there been a regression in the asm code to be incompatible with VC? Yes, SSE2 deblocking causes a crash with any compiler that doesn't correctly preserve stack alignment.

[kemuri-_9]

Well, thanks for that information, looks like I'll have to start getting familiar with mingw to compile x264....

[Underground78]

I think there is something strange with last builds :

E:\test>x264 --pass 1 --bitrate 500 --stats "rev899.stats" --bframes 16 --b-pyramid --direct auto --filter 0:0 --subme 1
--analyse none --me dia --threads auto --thread-input --progress --no-psnr --output NUL "1x01-p1.avs"
avis [info]: 704x528 @ 25.00 fps (2271 frames)
x264 [info]: using cpu capabilities: MMX2 SSE2Slow
x264 [info]: slice I:22 Avg QP:12.66 size: 93186:00:00
x264 [info]: slice P:961 Avg QP:13.12 size: 74215
x264 [info]: slice B:1288 Avg QP:14.75 size: 35506
x264 [info]: mb I I16..4: 11.2% 0.0% 88.8%
x264 [info]: mb P I16..4: 17.5% 0.0% 0.0% P16..4: 81.8% 0.0% 0.0% 0.0% 0.0% skip: 0.7%
x264 [info]: mb B I16..4: 1.7% 0.0% 0.0% B16..8: 54.0% 0.0% 0.0% direct:40.6% skip: 3.7%
x264 [info]: final ratefactor: -1.#J
x264 [info]: direct mvs spatial:99.8% temporal:0.2%
x264 [info]: SSIM Mean Y:0.9921801
x264 [info]: kb/s:10489.0

encoded 2271 frames, 57.93 fps, 10489.13 kb/s

with revision 889 I have this for the same source :

E:\test>x264_889 --pass 1 --bitrate 500 --stats "rev889.stats" --bframes 16 --b-pyramid --direct auto --filter 0:0 --subme 1
--analyse none --me dia --threads auto --thread-input --progress --no-psnr --output NUL "1x01-p1.avs"
avis [info]: 704x528 @ 25.00 fps (2271 frames)
x264 [info]: using cpu capabilities: MMX2 SSE2Slow
x264 [info]: slice I:22 Avg QP:32.91 size: 11528:00:00
x264 [info]: slice P:969 Avg QP:35.95 size: 3602
x264 [info]: slice B:1280 Avg QP:37.05 size: 830
x264 [info]: mb I I16..4: 52.0% 0.0% 48.0%
x264 [info]: mb P I16..4: 15.6% 0.0% 0.0% P16..4: 44.5% 0.0% 0.0% 0.0% 0.0% skip:39.9%
x264 [info]: mb B I16..4: 0.6% 0.0% 0.0% B16..8: 16.1% 0.0% 0.0% direct: 3.4% skip:79.9%
x264 [info]: final ratefactor: 31.72
x264 [info]: direct mvs spatial:86.6% temporal:13.4%
x264 [info]: SSIM Mean Y:0.8997162
x264 [info]: kb/s:423.3

encoded 2271 frames, 89.06 fps, 423.40 kb/s

Both builds are from x264.nl ... :scared:

[Dark Shikari]

:confused:

If you disable AQ, does the problem still occur? Does the problem occur with unpatched builds?

[Underground78]

:confused:

If you disable AQ, does the problem still occur?

Yes ...

Does the problem occur with unpatched builds?

Yes, tests were made with unpatched builds ...

[Dark Shikari]

Tested, works just fine here.x264 [info]: final ratefactor: -1.#JLooks like something is generating a floating point exception in ratecontrol. That would definitely cause the problem...

... any chance you could post an output stream, and try running it with --no-asm?

[Underground78]

Hum I will test with --no-asm, but output stream may be a little too big because of high bitrate ... I will test with a smaller sample.

If it helps, I could say that the bug appear in one of this revisions :

commit 16b8f79bb6825053b6dd0eebb1d24c1bdf112fbb r895
Author: Jason Garrett-Glaser <darkshikari@gmail.com>
Date: Wed Jul 2 10:43:57 2008 -0600

Fix bug in adaptive quantization
In some cases adaptive quantization did not correctly calculate the variance.
Bug reported by MasterNobody

commit 3440cf193fc13e558d2c6b98024d59e95a7b2774 r894
Author: Loren Merritt <pengvado@akuvian.org>
Date: Sun Jun 29 00:00:03 2008 -0600

lowres_init asm
rounding is changed for asm convenience. this makes the c version slower, but there's no way around that if all the implementations are to have the same results.

commit 1c8f807054a0308482d53d4c58ba1d5f5d2ae263 r893
Author: Jason Garrett-Glaser <darkshikari@gmail.com>
Date: Tue Jul 1 23:42:39 2008 -0600

Optimizations and cosmetics in macroblock.c
If an i4x4 dct block has no coefficients, don't bother with dequant/zigzag/idct. Not useful for larger sizes because the odds of an empty block are much lower.
Cosmetics in i16x16 to be more consistent with other similar functions.
Add an SSD threshold for chroma in probe_skip to improve speed and minimize time spent on chroma skip analysis.
Rename lambda arrays to lambda_tab for consistency.

[Underground78]

Ok, here a test with a smaller sample (note that with a 500 frames sample, the bug doesn't seem to appear) :

E:\film>x264 --pass 1 --bitrate 500 --stats "asm.stats" --bframes 16 --b-pyramid --direct auto --filter 0:0 --subme 1
--analyse none --me dia --threads auto --thread-input --progress --no-psnr --output "asm.264" "1x01-p1.avs"
avis [info]: 704x528 @ 25.00 fps (1001 frames)
x264 [info]: using cpu capabilities: MMX2 SSE2Slow
x264 [info]: slice I:12 Avg QP:12.66 size: 74475:00:00
x264 [info]: slice P:542 Avg QP:12.75 size: 60999
x264 [info]: slice B:447 Avg QP:14.26 size: 26072
x264 [info]: mb I I16..4: 15.1% 0.0% 84.9%
x264 [info]: mb P I16..4: 27.6% 0.0% 0.0% P16..4: 65.7% 0.0% 0.0% 0.0% 0.0% skip: 6.6%
x264 [info]: mb B I16..4: 3.8% 0.0% 0.0% B16..8: 43.2% 0.0% 0.0% direct:28.8% skip:24.2%
x264 [info]: final ratefactor: -1.#J
x264 [info]: direct mvs spatial:99.3% temporal:0.7%
x264 [info]: SSIM Mean Y:0.9930855
x264 [info]: kb/s:9112.7

encoded 1001 frames, 60.26 fps, 9112.89 kb/s

I've uploaded asm.264 (http://www.mediafire.com/?1lkjyymsnef) and asm.stats (http://www.mediafire.com/?uz141xyxtyi) ...

Without asm, the bug seems to disappear :

E:\film>x264 --pass 1 --bitrate 500 --stats "no-asm.stats" --bframes 16 --b-pyramid --direct auto --filter 0:0 --subme 1
--analyse none --me dia --no-asm --threads auto --thread-input --progress --no-psnr --output "no-asm.264" "1x01-p1.avs"

avis [info]: 704x528 @ 25.00 fps (1001 frames)
x264 [info]: using cpu capabilities: none!
x264 [info]: slice I:12 Avg QP:30.99 size: 11819:00:00
x264 [info]: slice P:542 Avg QP:33.32 size: 4207
x264 [info]: slice B:447 Avg QP:35.89 size: 861
x264 [info]: mb I I16..4: 47.3% 0.0% 52.7%
x264 [info]: mb P I16..4: 23.5% 0.0% 0.0% P16..4: 36.7% 0.0% 0.0% 0.0% 0.0% skip:39.8%
x264 [info]: mb B I16..4: 1.2% 0.0% 0.0% B16..8: 15.4% 0.0% 0.0% direct: 2.9% skip:80.5%
x264 [info]: final ratefactor: 31.95
x264 [info]: direct mvs spatial:94.6% temporal:5.4%
x264 [info]: SSIM Mean Y:0.9236085
x264 [info]: kb/s:560.7

encoded 1001 frames, 29.47 fps, 560.97 kb/s

[Dark Shikari]

I'm going to take a wild guess and say that the problem is the lowres_asm, and that when it runs using the MMX version, the FPU isn't re-initialized correctly afterwards (because we never had any MMX code this early in the program before).

The problem is that I can't duplicate it here even when disabling SSE assembly (and using only MMX).

[Underground78]

If you want me to do some tests, I would be happy to help ...

[Dark Shikari]

Sure, test time.

Download these builds (http://www.mediafire.com/?l1fbtgimwmx).

1. Test x264_old.exe. Does it exhibit the bug? If it doesn't, stop.
2. If it does exhibit the bug, test x264_new.exe. Does it exhibit the bug?

[squid_80]

The problem is that I can't duplicate it here even when disabling SSE assembly (and using only MMX).
Perhaps your machine is using SSE2 for floating point operations and doesn't give a crap about the FPU state?

[Dark Shikari]

Perhaps your machine is using SSE2 for floating point operations and doesn't give a crap about the FPU state?Nope, it won't do that unless you compile with a -march that allows SSE.

[jeffy]

x264 [info]: using cpu capabilities: MMX2 SSE2Slow

@Dark Shikari: could the error be originating from SSE2Slow?

I downloaded the file from the Mediafire and used the following script:
AVCSource("asm.dga")

C:\Program Files\megui\tools\x264>x264 --version
x264 0.60.895M 16b8f79

C:\Program Files\megui\tools\x264>x264 --pass 1 --bitrate 500 --stats "no-asm.s
ats" --bframes 16 --b-pyramid --direct auto --filter 0:0 --subme 1 --analyse no
e --me dia --no-asm --threads 2 --thread-input --progress --no-psnr --output "n
-asm.264" "c.avs"
avis [info]: 704x528 @ 25.00 fps (1001 frames)
x264 [info]: using cpu capabilities: none!
x264 [info]: slice I:12 Avg QP:30.86 size: 11873:00:00
x264 [info]: slice P:538 Avg QP:33.20 size: 4229
x264 [info]: slice B:451 Avg QP:35.21 size: 861
x264 [info]: mb I I16..4: 47.0% 0.0% 53.0%
x264 [info]: mb P I16..4: 23.8% 0.0% 0.0% P16..4: 37.1% 0.0% 0.0% 0.0%
.0% skip:39.0%
x264 [info]: mb B I16..4: 1.2% 0.0% 0.0% B16..8: 15.4% 0.0% 0.0% direct
2.9% skip:80.5%
x264 [info]: final ratefactor: 31.87
x264 [info]: direct mvs spatial:95.1% temporal:4.9%
x264 [info]: SSIM Mean Y:0.9258877
x264 [info]: kb/s:560.6

encoded 1001 frames, 28.88 fps, 560.80 kb/s

C:\Program Files\megui\tools\x264>x264 --pass 1 --bitrate 500 --stats "asm.stat
" --bframes 16 --b-pyramid --direct auto --filter 0:0 --subme 1 --analyse none
-me dia --threads 2 --thread-input --progress --no-psnr --output "asm.264" "c.avs"
avis [info]: 704x528 @ 25.00 fps (1001 frames)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 PHADD SSE4 Cache64
x264 [info]: slice I:12 Avg QP:30.86 size: 11873:00:00
x264 [info]: slice P:538 Avg QP:33.20 size: 4229
x264 [info]: slice B:451 Avg QP:35.21 size: 861
x264 [info]: mb I I16..4: 47.0% 0.0% 53.0%
x264 [info]: mb P I16..4: 23.8% 0.0% 0.0% P16..4: 37.1% 0.0% 0.0% 0.0%
.0% skip:39.0%
x264 [info]: mb B I16..4: 1.2% 0.0% 0.0% B16..8: 15.4% 0.0% 0.0% direct
2.9% skip:80.5%
x264 [info]: final ratefactor: 31.87
x264 [info]: direct mvs spatial:95.1% temporal:4.9%
x264 [info]: SSIM Mean Y:0.9258878
x264 [info]: kb/s:560.6

encoded 1001 frames, 78.03 fps, 560.80 kb/s

[Underground78]

Sure, test time.

Download these builds (http://www.mediafire.com/?l1fbtgimwmx).

1. Test x264_old.exe. Does it exhibit the bug? If it doesn't, stop.
2. If it does exhibit the bug, test x264_new.exe. Does it exhibit the bug?

x264_old.exe doesn't seem to exhibit the bug :

E:\film>x264_old --pass 1 --bitrate 500 --stats "x264_old.stats" --bframes 16 --b-pyramid --direct auto --filter 0:0 --subme 1
--analyse none --me dia --no-asm --threads auto --thread-input --progress --no-psnr --output "x264_old.264" "1x01-p1.avs"
avis [info]: 704x528 @ 25.00 fps (1001 frames)
x264 [info]: using cpu capabilities: none!
x264 [info]: slice I:10 Avg QP:30.26 size: 11348:00:00
x264 [info]: slice P:543 Avg QP:33.20 size: 4243
x264 [info]: slice B:448 Avg QP:35.50 size: 866
x264 [info]: mb I I16..4: 50.2% 0.0% 49.8%
x264 [info]: mb P I16..4: 23.6% 0.0% 0.0% P16..4: 37.0% 0.0% 0.0% 0.0% 0.0% skip:39.4%
x264 [info]: mb B I16..4: 1.2% 0.0% 0.0% B16..8: 15.2% 0.0% 0.0% direct: 3.0% skip:80.6%
x264 [info]: final ratefactor: 31.89
x264 [info]: direct mvs spatial:94.6% temporal:5.4%
x264 [info]: SSIM Mean Y:0.9242392
x264 [info]: kb/s:560.5

encoded 1001 frames, 16.00 fps, 560.69 kb/s

[jeffy]

x264_old.exe doesn't seem to exhibit the bug :

E:\film>x264_old --pass 1 --bitrate 500 --stats "x264_old.stats" --bframes 16 --b-pyramid --direct auto --filter 0:0 --subme 1
--analyse none --me dia --no-asm --threads auto --thread-input --progress --no-psnr --output "x264_old.264" "1x01-p1.avs"
avis [info]: 704x528 @ 25.00 fps (1001 frames)
x264 [info]: using cpu capabilities: none!
x264 [info]: slice I:10 Avg QP:30.26 size: 11348:00:00
x264 [info]: slice P:543 Avg QP:33.20 size: 4243
x264 [info]: slice B:448 Avg QP:35.50 size: 866
x264 [info]: mb I I16..4: 50.2% 0.0% 49.8%
x264 [info]: mb P I16..4: 23.6% 0.0% 0.0% P16..4: 37.0% 0.0% 0.0% 0.0% 0.0% skip:39.4%
x264 [info]: mb B I16..4: 1.2% 0.0% 0.0% B16..8: 15.2% 0.0% 0.0% direct: 3.0% skip:80.6%
x264 [info]: final ratefactor: 31.89
x264 [info]: direct mvs spatial:94.6% temporal:5.4%
x264 [info]: SSIM Mean Y:0.9242392
x264 [info]: kb/s:560.5

encoded 1001 frames, 16.00 fps, 560.69 kb/s

No wonder...

[Underground78]

Arg, my bad ! I will redo the tests ... :eek:

Edit : Hum, finally it doesn't seem to change anything, the bug doesn't seem to appear with x264_old.exe :

E:\film>x264_old --pass 1 --bitrate 500 --stats "x264_old.stats" --bframes 16 --b-pyramid --direct auto --filter 0:0 --subme 1
--analyse none --me dia --threads auto --thread-input --progress --no-psnr --output "x264_old.264" "1x01-p1.avs"

avis [info]: 704x528 @ 25.00 fps (1001 frames)
x264 [info]: using cpu capabilities: MMX2 SSE2Slow
x264 [info]: slice I:10 Avg QP:30.81 size: 11034:00:00
x264 [info]: slice P:544 Avg QP:33.20 size: 4240
x264 [info]: slice B:447 Avg QP:35.72 size: 865
x264 [info]: mb I I16..4: 50.9% 0.0% 49.1%
x264 [info]: mb P I16..4: 23.5% 0.0% 0.0% P16..4: 37.0% 0.0% 0.0% 0.0% 0.0% skip:39.5%
x264 [info]: mb B I16..4: 1.2% 0.0% 0.0% B16..8: 15.2% 0.0% 0.0% direct: 3.0% skip:80.6%
x264 [info]: final ratefactor: 31.89
x264 [info]: direct mvs spatial:94.9% temporal:5.1%
x264 [info]: SSIM Mean Y:0.9241641
x264 [info]: kb/s:560.2

encoded 1001 frames, 48.16 fps, 560.40 kb/s

[Dark Shikari]

Both x264_new and x264_old are my latest dev builds (one with an emms after the frame_init_lowres).

Since you don't get any problems with x264_old, there's a few possibilities:

1. Jarod's compiler sucks.
2. It relates to threads--my build doesn't have threads. Try disabling threads on jarod's builds and see if the problem goes away.

[Underground78]

2. It relates to threads--my build doesn't have threads. Try disabling threads on jarod's builds and see if the problem goes away.

Yes, it does disappear with --threads 1 ...

[Dark Shikari]

Yes, it does disappear with --threads 1 ...Try adding --asm SSE2 to the commandline on a build/commandline which otherwise breaks. Does the problem still occur?

[Underground78]

It seems that the bug doesn't occur if I add --asm SSE2 to the command line ...

E:\film>x264 --pass 1 --bitrate 500 --stats "threads-auto-asm-SSE2.stats" --bframes 16 --b-pyramid --direct auto --filter 0:0 --subme 1
--analyse none --me dia --asm SSE2 --threads auto --thread-input --progress --no-psnr --no-ssim --output "threads-auto-asm-SSE2.mkv" "1x01-p1.avs"
avis [info]: 704x528 @ 25.00 fps (1001 frames)
x264 [info]: using cpu capabilities: MMX2 SSE2
x264 [info]: slice I:12 Avg QP:30.99 size: 11820:00:00
x264 [info]: slice P:542 Avg QP:33.31 size: 4208
x264 [info]: slice B:447 Avg QP:35.87 size: 861
x264 [info]: mb I I16..4: 47.3% 0.0% 52.7%
x264 [info]: mb P I16..4: 23.5% 0.0% 0.0% P16..4: 36.7% 0.0% 0.0% 0.0% 0.0% skip:39.8%
x264 [info]: mb B I16..4: 1.2% 0.0% 0.0% B16..8: 15.5% 0.0% 0.0% direct: 2.9% skip:80.5%
x264 [info]: final ratefactor: 31.94
x264 [info]: direct mvs spatial:94.9% temporal:5.1%
x264 [info]: kb/s:560.9

encoded 1001 frames, 77.55 fps, 561.11 kb/s

[Dark Shikari]

Ding ding ding, we have a winner!

I'm going to guess its related to how frame_init_lowres interacts with ratecontrol in threaded mode.

If this is true, then the following changes will both solve the problem:

1. Disable frame_init_lowres asm (but don't disable anything else).

2. Add x264_emms() after the call to frame_init_lowres asm.

My build environment doesn't support threads, so I'll have to find someone to make you a few builds to test.

[Underground78]

My build environment doesn't support threads, so I'll have to find someone to make you a few builds to test.

I can do a build with threads support if you give me a patch to apply on GIT source code ...

[Dark Shikari]

I can do a build with threads support if you give me a patch to apply on GIT source code ...For idea 2 above, find the following code in encoder/encoder.c:

if( h->frames.b_have_lowres )
x264_frame_init_lowres( h, fenc );

Add, after it (on the next line) x264_emms();

For idea 1 above, find the following code in common/x86/mc-c.c:
pf->frame_init_lowres_core = x264_frame_init_lowres_core_mmxext;
pf->frame_init_lowres_core = x264_frame_init_lowres_core_cache32_mmxext;
pf->frame_init_lowres_core = x264_frame_init_lowres_core_cache32_mmxext;

Comment these out or remove them.

Test with no modifications, with just idea 1, and with just idea 2, and see if any of them resolve the issue.

[J_Darnley]

Here you go. 7-zip archive contains two builds with threads of the two ideas.
http://users.telenet.be/darnley/x264-lowres-ideas.7z

Dark_Shikari, you posted two identical lines. Is that correct? It does appear twice in mc-c.c for cacheline 32 and 64.

[Dark Shikari]

Here you go. 7-zip archive contains two builds with threads of the two ideas.
http://users.telenet.be/darnley/x264-lowres-ideas.7z

Dark_Shikari, you posted two identical lines. Is that correct? It does appear twice in mc-c.c for cacheline 32 and 64.Yes, that was correct.

It needs to be three builds, so that a comparison can be made with a normal build to see if that one is broken too.

[J_Darnley]

Ah, ok. I'll re-upload the archive. But I can say that for a first pass target of 500 the two ideas now give 515 instead of ~20000.
[EDIT] Three build archive uploaded.

[Underground78]

Sorry J_Darnley I've already made my test builds when you've posted but I was answering a phone call ... :o

Test builds gcc 3.4.5 fprofiled : Test builds (http://www.mediafire.com/?1njdjy1ti2y) ...

Default build :

E:\film>x264-default.exe --pass 1 --bitrate 500 --stats "default.stats" --bframes 16 --b-pyramid --direct auto --filter 0:0 --subme 1
--analyse none --me dia --threads auto --thread-input --progress --no-psnr --no-ssim --output "default.mkv" "1x01-p1.avs"
avis [info]: 704x528 @ 25.00 fps (1001 frames)
x264 [info]: using cpu capabilities: MMX2 SSE2Slow
x264 [info]: slice I:12 Avg QP:12.66 size: 74475:00:00
x264 [info]: slice P:542 Avg QP:12.75 size: 60999
x264 [info]: slice B:447 Avg QP:14.26 size: 26072
x264 [info]: mb I I16..4: 15.1% 0.0% 84.9%
x264 [info]: mb P I16..4: 27.6% 0.0% 0.0% P16..4: 65.7% 0.0% 0.0% 0.0% 0.0% skip: 6.6%
x264 [info]: mb B I16..4: 3.8% 0.0% 0.0% B16..8: 43.2% 0.0% 0.0% direct:28.8% skip:24.2%
x264 [info]: final ratefactor: -1.#J
x264 [info]: direct mvs spatial:99.3% temporal:0.7%
x264 [info]: kb/s:9112.7

encoded 1001 frames, 58.29 fps, 9112.89 kb/s

Idea 1 :

E:\film>x264-idea-1.exe --pass 1 --bitrate 500 --stats "idea-1.stats" --bframes 16 --b-pyramid --direct auto --filter 0:0 --subme 1
--analyse none --me dia --threads auto --thread-input --progress --no-psnr --no-ssim --output "idea-1.mkv" "1x01-p1.avs"
avis [info]: 704x528 @ 25.00 fps (1001 frames)
x264 [info]: using cpu capabilities: MMX2 SSE2Slow
x264 [info]: slice I:12 Avg QP:30.99 size: 11820:00:00
x264 [info]: slice P:542 Avg QP:33.31 size: 4208
x264 [info]: slice B:447 Avg QP:35.87 size: 861
x264 [info]: mb I I16..4: 47.3% 0.0% 52.7%
x264 [info]: mb P I16..4: 23.5% 0.0% 0.0% P16..4: 36.7% 0.0% 0.0% 0.0% 0.0% skip:39.8%
x264 [info]: mb B I16..4: 1.2% 0.0% 0.0% B16..8: 15.5% 0.0% 0.0% direct: 2.9% skip:80.5%
x264 [info]: final ratefactor: 31.94
x264 [info]: direct mvs spatial:94.9% temporal:5.1%
x264 [info]: kb/s:560.9

encoded 1001 frames, 87.76 fps, 561.11 kb/s

Idea 2 :

E:\film>x264-idea-2.exe --pass 1 --bitrate 500 --stats "idea-2.stats" --bframes 16 --b-pyramid --direct auto --filter 0:0 --subme 1
--analyse none --me dia --threads auto --thread-input --progress --no-psnr --no-ssim --output "idea-2.mkv" "
1x01-p1.avs"
avis [info]: 704x528 @ 25.00 fps (1001 frames)
x264 [info]: using cpu capabilities: MMX2 SSE2Slow
x264 [info]: slice I:12 Avg QP:30.99 size: 11820:00:00
x264 [info]: slice P:542 Avg QP:33.31 size: 4208
x264 [info]: slice B:447 Avg QP:35.87 size: 861
x264 [info]: mb I I16..4: 47.3% 0.0% 52.7%
x264 [info]: mb P I16..4: 23.5% 0.0% 0.0% P16..4: 36.7% 0.0% 0.0% 0.0% 0
.0% skip:39.8%
x264 [info]: mb B I16..4: 1.2% 0.0% 0.0% B16..8: 15.5% 0.0% 0.0% direct:
2.9% skip:80.5%
x264 [info]: final ratefactor: 31.94
x264 [info]: direct mvs spatial:94.9% temporal:5.1%
x264 [info]: kb/s:560.9

encoded 1001 frames, 90.10 fps, 561.11 kb/s

Both ideas 1 and 2 seem to work to fix the bug ...

[Dark Shikari]

Hopefully resolved (http://git.videolan.org/?p=x264.git;a=commit;h=a9af9425820cfba99ae4b378c33c4fee4e99b2ce).

[Underground78]

It works :

E:\film>x264-fixed --pass 1 --bitrate 500 --stats "fixed.stats" --bframes 16 --b-pyramid --direct auto --filter 0:0 --subme 1
--analyse none --me dia --threads auto --thread-input --progress --no-psnr --no-ssim --output "fixed.mkv" "1x01-p1.avs"

avis [info]: 704x528 @ 25.00 fps (1001 frames)
x264 [info]: using cpu capabilities: MMX2 SSE2Slow
x264 [info]: slice I:12 Avg QP:30.99 size: 11820:00:00
x264 [info]: slice P:542 Avg QP:33.31 size: 4208
x264 [info]: slice B:447 Avg QP:35.87 size: 861
x264 [info]: mb I I16..4: 47.3% 0.0% 52.7%
x264 [info]: mb P I16..4: 23.5% 0.0% 0.0% P16..4: 36.7% 0.0% 0.0% 0.0% 0.0% skip:39.8%
x264 [info]: mb B I16..4: 1.2% 0.0% 0.0% B16..8: 15.5% 0.0% 0.0% direct: 2.9% skip:80.5%
x264 [info]: final ratefactor: 31.94
x264 [info]: direct mvs spatial:94.9% temporal:5.1%
x264 [info]: kb/s:560.9

encoded 1001 frames, 91.65 fps, 561.11 kb/s

Thank you very much ! :)

[garbii]

I had to reverted to a x264 version from 2008 04 08 ... The one I was using right now was from 2008:12:30. The difference is ubntu jaunty vs ubuntu intrepid :P


Anyway new x264 has a worse CRF mode... although crf 22 gave me the old crf 20 quality, the size is actually larger ...
The settings for both encodes are the same with a difference that with the new one I removed bime and brdo from command line and instead use subme 7. (which is due to x264 api changes)

Old crf=20
minami-ke 08 size 42 MB
gundam 00 s2 20 size 80 MB

new crf=22
minami-ke 08 size 56 MB
gundam 00 s2 20 size 84 MB

Munto 05
old = 54 mb
new=70 mb

Now the difference on the same files shows that new crf mode is less efficient then the old one... So I consider this a bug since new version provides the same quality but with a lot larger files .

[Sharktooth]

that's not a bug but an intended behaviour.
talking about compression efficiency and CRF doesnt have much sense.
fact is, at the same filesize, the new x264 version gives better quality. hence, use the newer version and rise the CRF until you get your desirer filesize (and as i said, at the same filesize, you'll get a better quality).

[garbii]

that's not a bug but an intended behaviour.
talking about compression efficiency and CRF doesnt have much sense.
fact is, at the same filesize, the new x264 version gives better quality. hence, use the newer version and rise the CRF until you get your desirer filesize (and as i said, at the same filesize, you'll get a better quality).
I played with different numbers of CRF and the problem here is the differences in size. As you can see with new x264 the GUndam episode is quite close in size to what the old one was... The quality is also good. However less action packed shows like Munto and Minami are a lot larger then the old ones while having the same quality. THis is what I consider a bug because I found my desired quality (crf=22), but it doesn't seem to work as the old one did for non action shows.

[nm]

Old crf=20
minami-ke 08 size 42 MB
gundam 00 s2 20 size 80 MB

new crf=22
minami-ke 08 size 56 MB
gundam 00 s2 20 size 84 MB

Munto 05
old = 54 mb
new=70 mb

Now the difference on the same files shows that new crf mode is less efficient then the old one... So I consider this a bug since new version provides the same quality but with a lot larger files .
It would be better to do a 2-pass encode to the same size and then compare quality visually.

Looks like your source is anime. The 2008-04 version of x264 didn't have psy-RD, which may not be a good idea for animation. Now psy-RD is enabled by default, so try disabling it with --psy-rd 0.0:0.0 (or psy-rd=0.0,0.0 if you use MEncoder).

[Sharktooth]

I played with different numbers of CRF and the problem here is the differences in size. As you can see with new x264 the GUndam episode is quite close in size to what the old one was... The quality is also good. However less action packed shows like Munto and Minami are a lot larger then the old ones while having the same quality. THis is what I consider a bug because I found my desired quality (crf=22), but it doesn't seem to work as the old one did for non action shows.
again, this is not a bug but an intended behaviour.
as nm said, try disabling psy options since your encode is anime.

[garbii]

I enabled psy-rd=0.0,0.0 and gave new build a try I still have the same problem though... Let me rephrase. CRF should be a stable quality factor so once I choose the crf value I like it should always give me the same quality on files, or similiar at least. This is not the case with new crf, where I should consider the source and then choose the crf and if I have to do that I might as well use good old 2pass bitrate.

Extreme example of non action source vs a very demanding fast paced source:

Old crf=20
gundam 00 s2 ep 23 140 MB
mianami-ke okaeri 10 43 MB
both encodes have similiar quality. Both files are a 20 minutes shows.

New crf=24
gundam 00 s2 ep 23 81 MB
mianami-ke okaeri 10 45 MB
While the non action minami has good quality the 80 mb gundam encode just sucks in terms of quality.


New crf=22
gundam 00 s2 ep 23 100 MB
mianami-ke okaeri 10 53 MB
Quality wise minami seems the same as crf=24, it size is larger though ... Gundam is better then the previous encode but still not as good as the 140 MB encode which is just natural :)

I find it that new crf behavour makes this mode uselles for me. This example should really show that the previous version was better at keeping the same quality and not over enlarging filesize where not needed . With the old mode I could just set it to one value and not worry because all my encodes ended with similiar quality, with the new mode I would have to use 21 or maybe 20 for gundam and 24 for minami ... Now for me that's not what constant quality mode should be all about.

[Dark Shikari]

I enabled psy-rd=0.0,0.0 and give new build a try I still have the same problem though... Let me rephrase. CRF should be a stable quality factor so once I choose the crf value I like it should always give me the same quality on files, or similiar at least. This is not the case with new crf, where I should consider the source and then choose the crf and if I have to do that I might as well use good old 2pass bitrate.

Extreme example of non action source vs a very demanding fast paced source:

Old crf=20
gundam 00 s2 ep 23 140 MB
mianami-ke okaeri 10 43 MB
both encodes have similiar quality.

New crf=24
gundam 00 s2 ep 23 81 MB
mianami-ke okaeri 10 45 MB
While the non action minami has good quality the 80 mb gundam encode just sucks in terms of quality.


New crf=22
gundam 00 s2 ep 23 100 MB
mianami-ke okaeri 10 53 MB
Quality wise minami seems the same as crf=24, it size is larger though ... Gundam is better then the previous encode but still not as good as the 140 MB encode which is just natural :)

I find it that new crf behavour makes this mode uselles for me. This example should really show that the previous version was better at keeping the same quality and not over enlarging filesize where not needed . With the old mode I could just set it to one value and not worry because all my encodes ended with similiar quality, with the new mode I would have to use 22 or maybe 20 for gundam and 24 for minami ... Now for me that's not what quality mode should be all about.I had exactly the same problem with the old CRF--which was fixed with my modifications. My Touhou clips could reach CRF ~30-35 without noticeable degradation while BlackPearl looked crap at above ~24. So, to paraphrase you, "the old CRF is useless to me."

Unless you can come up with a magical algorithm for judging the human visual system, there is no guarantee that any CRF algorithm will give equivalent quality across all sources--in fact, it's almost guaranteed it will not.

Anyways, the proper approach here is not to blame CRF but to adjust qcomp until you are satisfied with the mapping between CRF and quality level.

Also, though you probably don't realize this, there have actually been three CRFs.

1. Original CRF (similar to current CRF)

2. CRF after AQ (qcomp completely disabled)

3. New CRF (most similar to original CRF)

Let me put this in a nice large font size here:
During 2), CRF was effectively broken. The "new" CRF merely restored the original behavior.

If you don't like this behavior, what you really want is qcomp=1, i.e. disabled.

[audyovydeo]

Also, though you probably don't realize this, there have actually been three CRFs.

1. Original CRF (similar to current CRF)

2. CRF after AQ (qcomp completely disabled)

3. New CRF (most similar to original CRF)



mmmh, interesting. I assume that :

1 = <r968
2 = r968
3 = >r????


cheers
audyovydeo

[nm]

mmmh, interesting. I assume that :

1 = <r968
2 = r968
3 = >r????
I'd guess something like this:

1: rev < r804 (variance-based psy adaptive quantization)
2: r804 <= rev < r968
3: rev >= r968 (move adaptive quantization to before ratecontrol, eliminate qcomp bias)

[garbii]

If you don't like this behavior, what you really want is qcomp=1, i.e. disabled.

Thanks that helped :) However just so you know it still not the same as the old "broken" crf. While it gives the same filesizes (and better quality) it's only up to some point . For example the eariler mentioned gundam ep with crf=22 and qcomp=1 ended up as a 102 MB file while with old value it was 140 mb ...

The quality of the mentioned ep is still quite good (not as good as the 140 mb one obviously) and I'm aiming to end up with small files anyway so I'm happy :)

Thank you for your help, and I still don't like how the normal non "broken" crf behaves ... oh well ...

[daimroc]

I get the error malloc size with de x264 r1201 (32 bits).

I have the following command line:

x264 --crf 16 --keyint 240 --min-keyint 24 --scenecut 40 --bframes 16 --b-adapt 2 --ref 4 --deblock 0:0 --qpmin 5 --qpmax 24 --aq-mode 1 --aq-strength 0.25 --mbtree --partitions all --direct auto --weightb --me tesa --merange 32 --mvrange -1 --mvrange-thread -1 --subme 9 --psy-rd 1.0:0.0 --mixed-refs --8x8dct --trellis 2 --no-fast-pskip --no-dct-decimate --nr 0 --threads auto --thread-input --output "video.mkv" "video.avs"



My avs is:

DirectShowSuorce("video.,m2ts")
Crop(Crop(0,132,-0,-132)

The source is a 1080p video, which I do a crop.

If I do a resize to 640x368 for example I don't have any problem.

I have a Core 2 Duo E6750 with 2GB of RAM, winth windows vista x64.




Thanks.
Daimroc.

[Dark Shikari]

That means your computer ran out of mappable memory for x264. Upgrade to 64-bit or stop using such insane settings.

[nm]

My avs is:

DirectShowSuorce("video.,m2ts")
Crop(Crop(0,132,-0,-132)
Try limiting AviSynth memory usage by adding SetMemorymax(32) to the script.

[VFR maniac]

Rev1201 mp4 output is broken, because the error is returned in write_nalu_mp4 though malloc has succeeded.

Here (http://www.esnips.com/doc/29a52597-d923-4ac0-96be-8e5b7ce14dff/x264_fix_mp4output_broken) is patch.

[CruNcher]

1201 crashes here :(

ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ
Transcoding Video: parkrun.avs / FPS: 25.000
ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ

avis [info]: 1280x720 @ 25.00 fps (489 frames)
x264 [info]: using cpu capabilities: MMX2 SSE2Slow
x264 [info]: profile High, level 3.1
mp4 [info]: initial delay 1 (scale 25)
[8.6%] 42/489 frames, 31.25 fps, 0.00 kb/s, eta 0:00:14


very simple params
x264.exe "%SourceVideo%" --crf 29 -o "%SourceVideo%.mp4"

yep happens only with .mp4 as output container

[kemuri-_9]

mp4 output should be working again in r1202, sorry for the 10L.

[bob0r]

mp4 output should be working again in r1202, sorry for the 10L.

10 Litres of Coke – Not Good Idea (http://a-h-z.com/cola-in-excess-can-paralyse/)

[bitbytebit]

Not sure if this is a bug or a major change in behavior, guessing it's some kind of memory leak though.

I have a program using x264 using the avcodec API, opens and closes the x264 encoder from there making 2 minute segments of video. Up to this version "x264 core:68 r1195 5d75a9b" memory usage stays at the same amount over 24 hours of opening/closing x264 (see in the ffmpeg libx264.c how it uses the open/close stuff to x264). When I try using the newer one (Macroblock-tree ratecontrol) and beyond with mbtree and rc_lookahead both on/off, then memory usage just climbs each close/open in ffmpeg avcodec of x264. This happens really fast with mbtree on, 4-5 meg per segmentation point, while even with both settings set to 0 (have altered ffmpeg libx264.c to use mbtree) it just does it slower and more like 1 meg per encoding. After a few hours this gets bad, while running the program with the older x264 it runs all day. Also can run the same program with any other codec from avcodec and doesn't budge in memory usage, just newer x264 using versions past the mbtree addition.

Not sure if it's ffmpeg using it wrong and some change caught that, have double checked and code seems like x264.c besides how they alloc the x4->pic structure. Seems like something may be the culprit in the mbtree stuff though, which looks awesome by the way and great for my encoding method (being very low bitrate), yet trying to get around this memory problem that occurs now. It works great on one shot programs, just when opened/closed multiple times seems to eat up memory till program exit.

Also using the newest git from just now, r1202, doesn't change the behavior I'm seeing (so seems to in the mbtree changes). Thanks for the hard work, mbtree is definitely an exciting feature.

[Chengbin]

mbtree EATS RAM. It is not a bug.

[Dark Shikari]

Not sure if this is a bug or a major change in behavior, guessing it's some kind of memory leak though.

I have a program using x264 using the avcodec API, opens and closes the x264 encoder from there making 2 minute segments of video. Up to this version "x264 core:68 r1195 5d75a9b" memory usage stays at the same amount over 24 hours of opening/closing x264 (see in the ffmpeg libx264.c how it uses the open/close stuff to x264). When I try using the newer one (Macroblock-tree ratecontrol) and beyond with mbtree and rc_lookahead both on/off, then memory usage just climbs each close/open in ffmpeg avcodec of x264. This happens really fast with mbtree on, 4-5 meg per segmentation point, while even with both settings set to 0 (have altered ffmpeg libx264.c to use mbtree) it just does it slower and more like 1 meg per encoding. After a few hours this gets bad, while running the program with the older x264 it runs all day. Also can run the same program with any other codec from avcodec and doesn't budge in memory usage, just newer x264 using versions past the mbtree addition.

Not sure if it's ffmpeg using it wrong and some change caught that, have double checked and code seems like x264.c besides how they alloc the x4->pic structure. Seems like something may be the culprit in the mbtree stuff though, which looks awesome by the way and great for my encoding method (being very low bitrate), yet trying to get around this memory problem that occurs now. It works great on one shot programs, just when opened/closed multiple times seems to eat up memory till program exit.

Also using the newest git from just now, r1202, doesn't change the behavior I'm seeing (so seems to in the mbtree changes). Thanks for the hard work, mbtree is definitely an exciting feature.Can you replicate the leak with Valgrind?

[bitbytebit]

Chengbin: It's with mbtree on or off actually.

Dark Shikari: I've tried, but it seems to not report any problems, not sure if I'm using it right....

valgrind -v --tool=memcheck --leak-check=yes --leak-resolution=high --show-reachable=yes --undef-value-errors=no PROGRAM

It always says everything was freed when I terminate the program, and exit really does the exact same stuff that each segmentation does...

/* free the streams */
for(i = 0; i < cs->oc->nb_streams; i++) {
avcodec_close(cs->oc->streams[i]->codec);
av_free(cs->oc->streams[i]->codec);
av_free(cs->oc->streams[i]);
}

// Free Muxer and Output Codecs
av_free(cs->oc);
cs->oc = NULL;
cs->vEncCtx = NULL;
cs->vEncCodec = NULL;
cs->video_st = NULL;
cs->audio_st = NULL;


So all that seems like it should free that x264 encoding instance, not sure what else would be hanging around still after all that.

[Dark Shikari]

Have you tried using libx264 directly instead of through libavcodec?

[bitbytebit]

That's actually my next idea I want to try, what I originally had ideally wanted but of course using libavcodec was easier and added muxing + easy video functions prewrapped. Doesn't look too daunting and would be nice to directly work with it, hoping that could fix it (although also worried it'll just be the same, since just changing x264 versions makes the libavcodec get the bad behavior, although it'd still be worth it either way getting x264 API usage direct in my program.).

[Dark Shikari]

Also, note that Valgrind should report at least one leak: x264 intentionally "leaks" a constant amount of statically allocated memory which is shared across all x264 processes and isn't re-allocated when you call x264 again (so it's not a leak over time); this is the few megabytes used for the MV cost arrays. If valgrind works correctly, it should whine about these, generally.

[bitbytebit]

Strange, it's not saying anything wrong for me, valgrind seems too happy then (in both newer/older x264 library)...

==8565==
==8565== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
==8565== malloc/free: in use at exit: 0 bytes in 0 blocks.
==8565== malloc/free: 0 allocs, 0 frees, 0 bytes allocated.
==8565== For counts of detected errors, rerun with: -v
==8565== All heap blocks were freed -- no leaks are possible.


Not sure how outputs happening then, seems like it should see the intentional leak, unless it's not able to see beyond avcodec (although isn't it supposed to see everything, I've got it all compiled with debugging too).

[bitbytebit]

Figured out the valgrind problem, now it's reporting things. The first is the previous x264 before mbtree and the second result is after (have the full output, with full debug on so shows files+line numbers, let me know if you want that and how to get it to you)....

x264 before mbtree:

==9092== LEAK SUMMARY:
==9092== definitely lost: 764 bytes in 30 blocks.
==9092== indirectly lost: 1,632 bytes in 78 blocks.
==9092== possibly lost: 1,644,994 bytes in 38 blocks.
==9092== still reachable: 11,114,785 bytes in 797 blocks.
==9092== suppressed: 0 bytes in 0 blocks.

Current GIT x264:
==9362== LEAK SUMMARY:
==9362== definitely lost: 4,671,872 bytes in 10,297 blocks.
==9362== indirectly lost: 2,630,324 bytes in 3,669 blocks.
==9362== possibly lost: 7,377,089 bytes in 291 blocks.
==9362== still reachable: 2,957,742 bytes in 604 blocks.
==9362== suppressed: 0 bytes in 0 blocks.


This is from running my program in 10 loops of segmentation (1 second ones so it didn't take all day being slow under valgrind).

[Dark Shikari]

Figured out the valgrind problem, now it's reporting things. The first is the previous x264 before mbtree and the second result is after (have the full output, with full debug on so shows files+line numbers, let me know if you want that and how to get it to you)....pastebin.

[bitbytebit]

Here's the output.... http://pastebin.com/m13806d41

Here's the exact info of what options setup in x264.... http://pastebin.com/m1d5b5a12

[Dark Shikari]

Fixed.

[bitbytebit]

Great, my program is no longer eating more and more memory over time, thanks for the quick fixing of this.

[daimroc]

I tried to use the command SetMemorymax(32) in the avs script but it doesn.t solve the problem.

I try to use the 64 bits versión of the x264, but I have problems with the avs2yuv. Although I always use this settings and I don't have any problem with r1195 of the 32 bits version of x264. I only have this problems with this new builds.

I use the same avs, and the command line is:

avs2yuv -raw "video.avs" - | "x264.exe" --crf 16 --keyint 240 --min-keyint 24 --scenecut 40 --bframes 16 --b-adapt 2 --ref 4 --deblock 0:0 --qpmin 5 --qpmax 24 --aq-mode 1 --aq-strength 0.25 --mbtree --partitions all --direct auto --weightb --me tesa --merange 32 --mvrange -1 --mvrange-thread -1 --subme 9 --psy-rd 1.0:0.0 --mixed-refs --8x8dct --trellis 2 --no-fast-pskip --no-dct-decimate --nr 0 --threads auto --output "video.mkv" - 1920x816

But I get the following error:

Video returned: "DirectShowSource: Renderfile, the filter graph manager won't talk to me".

What I do wrong?




Thanks.
Daimroc.

[thewebchat]

My guess is that the error has nothing to do with x264 but rather to do with this:


DirectShowSuorce("video.,m2ts")
Crop(Crop(0,132,-0,-132)

[daimroc]

Well, in the first post I made a mistake with the copy paste.

I have the following lines in the AVS:

DirectShowSource("00000.m2ts", audio=false)
Crop(0,132,-0,-132)



Thanks.
Daimroc.

[OvejaNegra]

the whole story:

my neighbour has the whole collection of 007 movies (all of them, except q of solace, but that one is on the way).


he ask me if i can do the conversion to mkv with the usuall stuff, i say yes because that way i can watch all the
007 movies (therea are a lot of them)

one of the DVDs started to show a lot of small white spots (maybe fungus) 2 or 3 years ago, so he made a recompressed
backup of the whole DVD with all the extras and audio tracks, so quality is totally destroyed on that one (goldfinger)


i'm trying to save it with this script (the second pass of two passes for tivtc):


DGDecode_mpeg2source("dgp.d2v", info=3,cpu=6)

ColorMatrix(hints=true, interlaced=true, threads=0)


interp = separatefields().EEDI2(field=-2)
px = tdeint(type=2,edeint=interp,mode=2,full=true)


#matching paso2
TFM (order=-1,mode=0,pp=5,display=false,clip2=px,slow=2,cthresh=9,d2v="dgp.d2v",input="TFM_matches.txt",ovr="ovr.txt",batch=true)


#decimating paso2
TDecimate (mode=5,hybrid=2,display=false,batch=true,input="TD_metrics.txt",tfmIn="TFM_matches.txt",mkvOut="tiempos.txt")

crop(8,0,704,480)



#from GK
Temporalsoften(3,5,5,mode=2,scenechange=10)
Convolution3d("moviehq")
FluxSmoothST(7,7)

#end


now this is what i'm using for all the collection without problems (of course with different filters)

this time, is the first time i'm using deblocking with DGDECODE

when i start the compression, on the first pass, x264 crash at 40% approx.


same hardware, no recent changes on eviroment (new software or anything like that).

i ran a analisys pass with megui without problems (the full movie) and my last attempt was direct with the cli encoder
(no megui)


now:
WXP SP3 32bits
DualCore Intel Pentium E2160, 1800 MHz (9 x 200)
MB Intel Coconut Creek D945GCCR (2 PCI, 1 PCI-E x1, 1 PCI-E x16, 2 DDR2 DIMM, Audio, Video, LAN)
Chipset Intel Lakeport-G i945GC
DIMM1: Kingston 9905316-005.A04LF 1 GB DDR2-667 DDR2 SDRAM (5-5-5-15 @ 333 MHz) (4-4-4-12 @ 266 MHz) (3-3-3-9 @ 200 MHz)

X264 cli output:

x264.exe --profile high --pass 1 --bitr
ate 1653 --stats ".stats" --ref 1 --no-mixed-refs --bframes 2 --b-adapt 2 --dire
ct auto --subme 2 --trellis 0 --partitions none --qpmin 12 --vbv-init 1.0 --me d
ia --thread-input --output NUL "e:\007\video proc\p2.avs"
avis [info]: 704x480 @ 23.98 fps (158257 frames)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile Main, level 3.0
[56.3%] 89050/158257 frames, 4.47 fps, 1652.03 kb/s, eta 4:18:13
"../kmp_runtime.c", line 5235: OMP runtime internal error: assertion failure.

Please submit a bug report with this message, compile and run commands used,
and machine configuration info including native compiler and operating system
versions. Faster response will be obtained by including all program sources.
Please send submissions to http://premier.intel.com/

PD you can see the low fps? well is not true, the frame counter runs faster than that

x264 core:70 r1203 6f4054f (but using the build i had before does not fix the problem)

i continue using my pc without problems, gaming, watching videos, music, everything seems ok, no perfomance loss or crashes on my aplications

if anybody needs more info like the stats, just ask
im using the last version for all my AVS filters and AVS 258

thanks

[OvejaNegra]

i just disabled macroblock tree ratecontrol.

everything is just fine now

[ChaosKing]

I've just ran some test encodes and then got this error: malloc of size 3211264 failed/s
And it seems that x264 dont like '%' in filenames.

x264 core:70 r1206 01a693d

start /b /w x264.exe --rc-lookahead 200 --preset placebo --crf 26 --keyint 240 --thread-input --output "ichi.mkv" "Ichigo.avs"
avis [info]: 960x720 @ 23.98 fps (1120 frames)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 FastShuffle SSE4.1 Cache64
x264 [info]: profile High, level 5.0
x264 [error]: malloc of size 3211264 failed/s, eta 0:00:21
x264 [error]: x264_encoder_encode failed

[Underground78]

I think this answer probably applies to your malloc error :

That means your computer ran out of mappable memory for x264. Upgrade to 64-bit or stop using such insane settings.

[ChaosKing]

thx, 64bit works without problems

[juGGaKNot]

after using bframes 0 and going back to bframes 3 cli crashes at the last frame in the first pass, 4 times in a row

set useless=--level %mylevel% --bitrate %btratex264% --stats "%mypath%\%mpath%\T1\movie.stats" --keyint %kint% --min-keyint %mint% --fullrange on --ref %myrefs% --bframes 3 --ratetol 2.0 --merange 32 --direct temporal --aq-mode 2 --no-fast-pskip --no-dct-decimate --vbv-bufsize 20000 --vbv-maxrate 20000 --nal-hrd --sar 1:1 --aud
echo.
echo %useless%
echo.
start "encode" /b /low /wait "%mypath%\bin\x264.exe" --pass 1 --preset veryslow %useless% --slow-firstpass --output NUL %myavs%
echo.
echo %langi%
echo.
start "encode" /b /low /wait "%mypath%\bin\x264.exe" --pass 2 --preset veryslow %useless% --output "%mypath%\%mpath%\T1\%mymovie%.264" %myavs%

same settings worked before with bframes 3, worked with bframes 0 not that i went back to bframes 3 it crashes

Encoding settings : cabac=1 / ref=5 / deblock=1:0:0 / analyse=0x3:0x133 / me=umh / subme=10 / psy=1 / psy_rd=1.0:0.0 / mixed_ref=1 / me_range=32 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / chroma_qp_offset=-2 / threads=3 / nr=0 / decimate=0 / mbaff=0 / bframes=0 / keyint=300 / keyint_min=30 / scenecut=40 / rc_lookahead=60 / rc=2pass / mbtree=1 / bitrate=4000 / ratetol=2.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / vbv_maxrate=20000 / vbv_bufsize=20000 / ip_ratio=1.40 / aq=2:1.00

i will edit with in 10 minutes with the cli log

[Dark Shikari]

GDB backtrace? Does it occur with YUV input?

[juGGaKNot]

avs

echo AVIsource("%mypath%\%mymovie%.avi") >> "%mypath%\%mpath%\T3\%mymovie%.avs"
echo ConvertToYV12(matrix="%rangepc%") >> "%mypath%\%mpath%\T3\%mymovie%.avs"
echo LoadPlugin("%mypath%\bin\autocrop.dll") >> "%mypath%\%mpath%\T3\%mymovie%.avs"
echo AutoCrop(0, 16, 16, threshold=0) >> "%mypath%\%mpath%\T3\%mymovie%.avs"

1184x666 uncompressed rgb24 with autocrop = 1184x656, matrix is PC.601

log in 4 minutes

Installing GDB.


avis [info]: 1184x656 @ 30.00 fps (2477 frames)
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Slow
x264 [info]: profile High, level 3.2
[100.0%] 2476/2477 frames, 1.54 fps, 3852.15 kb/s, eta 0:00:00

Encoding X264 2nd Pass :

avis [info]: 1184x656 @ 30.00 fps (2477 frames)
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Slow
x264 [error]: ratecontrol_init: can't open stats file
x264 [error]: x264_encoder_open failed

[juGGaKNot]

same stuff but --frames 120 works.

$ gdb /d/x264.exe
GNU gdb 5.2.1
Copyright 2002 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i686-pc-mingw32"...c:/cygmnt/prj/pkg/src/gdb/mingw32
/gdb/dwarf2read.c:985: gdb-internal-error: read_comp_unit_head: dwarf from non e
lf fil.

[Dark Shikari]

You still haven't answered my question.

[kemuri-_9]

and you're using a broken gdb, use release candidate 6.8-3

[juGGaKNot]

Updated to latest git, x264_hrd_pd_interlace.16.diff & x264_win_zone_parse_fix_06.diff, ./configure, fprofiled.

AVIsource("C:\x264\AVS2YUV.avi")
ConvertToYV12(matrix="PC.601")
LoadPlugin("C:\x264\bin\autocrop.dll")
AutoCrop(0, 16, 16, threshold=0)

AVS2YUV.exe AVS2YUV.avs -raw -o AVS2YUV.yuv

--preset veryslow --level %mylevel% --bitrate %btratex264% --stats "%mypath%\%mpath%\T1\movie.stats" --keyint %kint% --min-keyint %mint% --fullrange on --ref %myrefs% --bframes 0 --ratetol 2.0 --merange 32 --direct temporal --aq-mode 2 --no-fast-pskip --no-dct-decimate --vbv-bufsize 20000 --vbv-maxrate 20000 --nal-hrd --sar 1:1 --aud

avs bframes 0 : works
avs bframes 3 : crashes before last frame 2476/2477
yuv bframes 0 : works
yuv bframes 3 : works

TO DO :
updated to latest git, x264_hrd_pd_interlace.16.diff & x264_win_zone_parse_fix_06.diff, ./configure --enable-debug, fprofiled.

gdb /d/x264.exe
set args --preset veryslow --level 3.2 --bitrate 4000 --keyint 30 --min-keyint 30 --fullrange on --ref 5 --bframes 0 --ratetol 2.0 --merange 32 --direct temporal --aq-mode 2 --no-fast-pskip --no-dct-decimate --vbv-bufsize 20000 --vbv-maxrate 20000 --nal-hrd --sar 1:1 --aud
run
(wait for crash)
(post output)
bt
(post output)
disass $pc-30 $pc+30
(post this too)

./configure --enable-debug
gdb /d/x264.exe
set args --preset veryslow --level 3.2 --bitrate 4000 --keyint 30 --min-keyint 30 --fullrange on --ref 5 --bframes 3 --ratetol 2.0 --merange 32 --direct temporal --aq-mode 2 --no-fast-pskip --no-dct-decimate --vbv-bufsize 20000 --vbv-maxrate 20000 --nal-hrd --sar 1:1 --aud
run
(wait for crash)
(post output)
bt
(post output)
disass $pc-30 $pc+30
(post this too)

as soon as lord_mulder is on ... you know, you can take a n00b to water but you can't teach a noob how to drink.

[kemuri-_9]

as soon as lord_mulder is on ... you know, you can take a n00b to water but you can't teach a noob how to drink.

do what? what do you need LoRd_MuldeR for? you can do as we asked and that's that.

if you aren't going to post a bt we can't help the situation, but as you pointed at the it crashed with the .avs version of the source and not the .yuv,
the blame is likely with avisynth

don't post an entire post of '__TO_FILL_IN__', it's a waste of time to go through

[juGGaKNot]

i was waiting for help with gdb


Program exited normally.
(gdb)

gdb /d/x264.exe

set args --preset veryslow --level 3.2 --bitrate 4000 --frames 120 --keyint 30 --min-keyint 30 --fullrange on --ref 5 --bframes 0 --ratetol 2.0 --merange 32 --direct temporal --aq-mode 2 --no-fast-pskip --no-dct-decimate --vbv-bufsize 20000 --vbv-maxrate 20000 --nal-hrd --sar 1:1 --aud --output null d:/avs.avs

run

[kemuri-_9]

i was waiting for help with gdb

what help do you need? you have the entire sequence of commands to run to produce the bt...

Program exited normally.
we only want the bt for the case in which it crashes.
if it doesn't crash, it's just a waste of space to post the gdb output.

[juGGaKNot]

what help do you need? you have the entire sequence of commands to run to produce the bt...

Program exited normally.
we only want the bt for the case in which it crashes.
if it doesn't crash, it's just a waste of space to post the gdb output.

i had the old version, when you said get the new one i had to go, i came back ~20 minutes ago.

so there is no problem ? should i reinstall avysinth ?

[Revgen]

I'm doing some test encodes using Ben Waggoner's Island trailer he posted in the BD replication thread.

I've been having VBV underflow issues using 1222 build of x264 using these general settings.

x264 --profile high --level 4.1 --ssim --pass 2 --bitrate 40000 --stats "E:\King.stats" --thread-input --deblock -2:-2 --keyint 24 --min-keyint 2 --b-adapt 2 --direct auto --ref 4 --ipratio 1.1 --pbratio 1.1 --vbv-bufsize 30000 --vbv-maxrate 40000 --qcomp 0.5 --me umh --subme 9 --psy-rd 1.0:0.15 --partitions all --trellis 2 --no-mixed-refs --mvrange 511 --nal-hrd --aud --sar 1:1 --fullrange off --colorprim bt709 --colormatrix bt709 --transfer bt709 -o "g:\island.264" "g:\island.avs"

Using Jeeb's patched 1222 32-bit build, I always get 1 vbv error near the end of the encode. It doesn't matter if I use mbtree or not or use nal-hrd/aud or not. Komisar's generic 32-bit 1222 build doesn't give an error when I don't use mbtree, but it does give me one when I do use it.

[Dark Shikari]

Find a minimal set of settings that reproduces the problem, including minimizing the number of threads, if possible. Then, find the minimal number of frames necessary to replicate the problem and upload that specific section.

[Revgen]

Okay, I'll let you know what I find.

[Revgen]

After a lot of testing and head scratching, it seems that the vbv issues are being caused by --b-adapt 2. However, sometimes I can use --b-adapt 2 and it works just fine. Using --b-adapt 1 in the first pass and 2nd pass never results in vbv errors.

The vbv errors always happen in the last 419 frames.

This section is 474mb, so it'll take me awhile to upload it.

[Dark Shikari]

After a lot of testing and head scratching, it seems that the vbv issues are being caused by --b-adapt 2. However, sometimes I can use --b-adapt 2 and it works just fine. Using --b-adapt 1 in the first pass and 2nd pass never results in vbv errors.

The vbv errors always happen in the last 419 frames. These frames are 474mb, so it'll take me awhile to upload it.You can recompress it in x264 lossless if it helps.

By "last 419 frames", you mean that if you encode only those 419 frames, the error still happens, right (not that if you encode all the frames, the errors happen in the last 419 frames)?

Also, can you post the minimal commandline you found that produces the issue? If it only happens with --b-adapt 2, I suspect a bug in VBV lookahead.

[Revgen]

^Yep, only in the last 419 frames. I can encode those frames alone and get the errors.


Minimal Settings.

1st pass

x264 --profile high --level 4.1 --ssim --pass 1 --bitrate 40000 --stats "E:\King.stats" --thread-input --deblock -2:-2 --keyint 24 --min-keyint 2 --b-adapt 2 --direct auto --ref 4 --ipratio 1.1 --pbratio 1.1 --vbv-bufsize 30000 --vbv-maxrate 40000 --qcomp 0.5 --me dia --subme 2 --partitions none --trellis 0 --no-mixed-refs --mvrange 511 --nal-hrd --aud --sar 1:1 -o "g:\island7.264" "g:\island.avs"

2nd pass

x264 --profile high --level 4.1 --ssim --pass 2 --bitrate 40000 --stats "E:\King.stats" --thread-input --deblock -2:-2 --keyint 24 --min-keyint 2 --b-adapt 2 --direct auto --ref 4 --ipratio 1.1 --pbratio 1.1 --vbv-bufsize 30000 --vbv-maxrate 40000 --qcomp 0.5 --me umh --subme 9 --psy-rd 1.0:0.15 --partitions all --trellis 2 --no-mixed-refs --mvrange 511 --nal-hrd --aud --sar 1:1 --fullrange off --colorprim bt709 --colormatrix bt709 --transfer bt709 -o "g:\island7.264" "g:\island.avs"

[Dark Shikari]

Wait, it underflows on the first pass, second pass, or both?

And those aren't minimal settings--minimal means that you've tried to take out as many options as you can without causing it to stop underflowing. More options makes it harder for me to figure out what's causing the problem. I can do it myself, of course, but I can't get started until I have said 419 frames, so the more I know before I start working, the faster I can pinpoint the problem.

[Revgen]

Underflows on 2nd pass.

When you say minimalist settings, you mean when can I use --b-adapt 2 without causing errors? I'll look into that.

[Dark Shikari]

Underflows on 2nd pass.

When you say minimalist settings, you mean when can I use --b-adapt 2 without causing errors? I'll look into that.I mean you drop every single setting that doesn't have any relation to the problem.

So you drop a setting, check for underflow--if the underflow goes away, put the setting back. If it doesn't go away, drop the setting. Repeat until the commandlines are as short as possible.

A second pass underflow is very interesting; it means there's a bug in Gabriel's 2-pass VBV algorithm, not the VBV lookahead.

[Revgen]

Here's the last 419 frames.

http://www.megaupload.com/?d=RM8TF0R0


Finding minimal settings is difficult. Sometimes the settings bring vbv errors. Sometimes they don't. I have to run a test 3 times to make sure. The only certainty I get is that no matter what, --b-adapt 2 causes vbv errors sometimes and --b-adapt 1 never does. The rest is a mystery. I'll try to find the most minimal that I can, but I can't guarantee that they'll be exact.

[Dark Shikari]

Here's the last 419 frames.

http://www.megaupload.com/?d=RM8TF0R0


Finding minimal settings is difficult. Sometimes the settings bring vbv errors. Sometimes they don't. I have to run a test 3 times to make sure. The only certainty I get is that no matter what, --b-adapt 2 causes vbv errors sometimes and --b-adapt 1 never does. The rest is a mystery. I'll try to find the most minimal that I can, but I can't guarantee that they'll be exact.Downloading; I'll see what I can do.

[Dark Shikari]

I've run the settings a few times and I can't seem to get any VBV underflow to occur, using 3 threads on a Core 2 Duo.

[Revgen]

My cpu is a quad-core. I'll try 2 threads again and see what happens.

[Revgen]

Still getting errors.

I guess it's a problem on my end.

[Boolsheet]

If I try to encode the last 419 frames of The Island trailer I get those warnings too (late edit: Yes, only in second pass):
"x264 [warning]: VBV underflow (-75256 bits)"

They showed up with these settings:
--pass 1 and 2 (Same settings for both)
--b-adapt 0 or 2 (like Revgen I didn't get warnings with 1)
--bitrate 40000 --vbv-bufsize 30000 --vbv-maxrate 40000
--threads 8 or higher (I'm on a Core i7 HT on - makes 8 threads)

There are no warnings with fewer threads, no bframes or no-mbtree. I'm guessing it's those white frames beginning at frame 285 that causes problems.
Can the lookahead be disabled? It appears --rc-lookahead 0 does not have any effect. ;)

(I used JEEB's modified 1222 x64 build)

[Revgen]

I'm not alone afterall! :D

Now if only we can get a quad-core or better shipped out to DS or Aku.

Using less threads still gets me errors, but it takes a lot more tries to get it to happen. Most times it doesn't happen at all.

[Boolsheet]

Using less threads still gets me errors, but it takes a lot more tries to get it to happen. Most times it doesn't happen at all.
You're right, now I got 1 underflow with 7 threads. Maybe the frame-based threaded encoding gets sometimes the right frames and sometimes not.

[Sagittaire]

--bitrate 40000 --vbv-bufsize 30000 --vbv-maxrate 40000

Not really a bug with bitrate at max bitrate limite. With that you make buffering CBR encoding (aka VBR with only variable bitrate from buffer). It's really stressing setting for VBV.

You can probaly obtain better overall quality with --bitrate 30000 --vbv-bufsize 30000 --vbv-maxrate 40000 simply because encoder will have more buffer for really high complexity part. Higher is not always better and in this particular example encoding at 30 Mbps will certainely produce better quality than 40 Mbps encoding.

[Dark Shikari]

Hah, I ran it with 6 threads and I was able to replicate it. More analysis tomorrow :)

In general, underflows with 2-pass VBV should NEVER EVER HAPPEN, so bug reports like this are very important.

@Boolsheet, are you getting underflows in the first pass, or just second?

[Dark Shikari]

... and it seems I completely forgot, when writing VBV lookahead and MB-tree, that when VBV is on, the second pass still runs rc_analyse_slice. I guess it's a testament to how good Gabriel's algorithm is that it despite my stupid mistake... ;)

It seems that the naive fix doesn't exactly work though, so there might be a bit more to this.

Edit: .... and fixed locally. Commit coming today.

[Revgen]

You can probaly obtain better overall quality with --bitrate 30000 --vbv-bufsize 30000 --vbv-maxrate 40000 simply because encoder will have more buffer for really high complexity part. Higher is not always better and in this particular example encoding at 30 Mbps will certainely produce better quality than 40 Mbps encoding.

That's why this is an X264 Bug thread. :)

Edit: .... and fixed locally. Commit coming today.

Nice. Glad we could help. This Island trailer has managed to cause bugs in 2 programs already.

[Revgen]

Used Komisar's new generic build 1232, and I'm still getting the same VBV errors.

[Dark Shikari]

Used Komisar's new generic build 1232, and I'm still getting the same VBV errors.With pass 1 or 2?

Again, I need specifics, because I can't replicate the issue anymore.

[Revgen]

Pass 2. Same issues as before. Same 419 frames.

[Dark Shikari]

Settings used:

x264 --profile high --level 4.1 --ssim --pass 1 --bitrate 40000 --thread-input --deblock -2:-2 --keyint 24 --min-keyint 2 --b-adapt 2 --direct auto --ref 4 --ipratio 1.1 --pbratio 1.1 --vbv-bufsize 30000 --vbv-maxrate 40000 --qcomp 0.5 --me dia --subme 2 --partitions none --trellis 0 --no-mixed-refs --mvrange 511 --aud --sar 1:1 -o encode.h264 input.avs --threads 8

x264 --profile high --level 4.1 --ssim --pass 2 --bitrate 40000 --thread-input --deblock -2:-2 --keyint 24 --min-keyint 2 --b-adapt 2 --direct auto --ref 4 --ipratio 1.1 --pbratio 1.1 --vbv-bufsize 30000 --vbv-maxrate 40000 --qcomp 0.5 --me umh --subme 9 --psy-rd 1.0:0.15 --partitions all --trellis 2 --no-mixed-refs --mvrange 511 --aud --sar 1:1 --fullrange off --colorprim bt709 --colormatrix bt709 --transfer bt709 -o encode.h264 input.avs --threads 8 --verboseRan twice, no VBV issues.

[ajp_anton]

Using 1232 from x264.nl:

C:\Users\ajp_anton\Desktop\test>x264 --pass 1 --b-adapt 2 --bitrate 40000 --vbv-bufsize 30000 --vbv-maxrate 40000 --threads 12 -o NUL test.avs
avis [info]: 1920x1080 @ 23.98 fps (419 frames)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 FastShuffle SSE4.2
x264 [info]: profile Main, level 4.1
x264 [info]: slice I:39 Avg QP:13.22 size:248741
x264 [info]: slice P:168 Avg QP:16.17 size:208505
x264 [info]: slice B:212 Avg QP:16.59 size:193248
x264 [info]: consecutive B-frames: 12.6% 28.4% 31.6% 27.4%
x264 [info]: mb I I16..4: 63.4% 0.0% 36.6%
x264 [info]: mb P I16..4: 71.7% 0.0% 0.0% P16..4: 27.7% 0.0% 0.0% 0.0% 0.0% skip: 0.5%
x264 [info]: mb B I16..4: 35.3% 0.0% 0.0% B16..8: 28.6% 0.0% 0.0% direct:34.0% skip: 2.0% L0:26.6% L1:26.7% BI:46.7%
x264 [info]: coded y,uvDC,uvAC intra:80.4% 94.9% 67.3% inter:68.4% 80.5% 15.5%
x264 [info]: kb/s:39230.6

encoded 419 frames, 9.68 fps, 39231.15 kb/s

C:\Users\ajp_anton\Desktop\test>x264 --pass 2 --b-adapt 2 --bitrate 40000 --vbv-bufsize 30000 --vbv-maxrate 40000 --threads 12 -o NUL test.avs
avis [info]: 1920x1080 @ 23.98 fps (419 frames)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 FastShuffle SSE4.2
x264 [info]: profile High, level 4.1
x264 [warning]: VBV underflow (-1161622 bits)/s, eta 0:00:13
x264 [warning]: VBV underflow (-641179 bits)b/s, eta 0:00:13
x264 [warning]: VBV underflow (-645571 bits)b/s, eta 0:00:13
x264 [warning]: VBV underflow (-647619 bits)b/s, eta 0:00:13
x264 [warning]: VBV underflow (-36379 bits)kb/s, eta 0:00:12
x264 [warning]: VBV underflow (-344443 bits)b/s, eta 0:00:12
x264 [warning]: VBV underflow (-34931 bits)kb/s, eta 0:00:12
x264 [warning]: VBV underflow (-41579 bits)kb/s, eta 0:00:12
x264 [warning]: VBV underflow (-481859 bits)b/s, eta 0:00:12
x264 [warning]: VBV underflow (-34987 bits)kb/s, eta 0:00:12
x264 [warning]: VBV underflow (-20115 bits)kb/s, eta 0:00:12
x264 [warning]: VBV underflow (-206035 bits)b/s, eta 0:00:12
x264 [warning]: VBV underflow (-46475 bits)kb/s, eta 0:00:11
x264 [warning]: VBV underflow (-43891 bits)kb/s, eta 0:00:11
x264 [info]: slice I:39 Avg QP:15.56 size:210048
x264 [info]: slice P:168 Avg QP:16.75 size:216575
x264 [info]: slice B:212 Avg QP:16.90 size:201904
x264 [info]: consecutive B-frames: 12.6% 28.4% 31.6% 27.4%
x264 [info]: mb I I16..4: 17.9% 49.5% 32.7%
x264 [info]: mb P I16..4: 3.5% 39.3% 6.9% P16..4: 23.1% 13.2% 12.6% 0.0% 0.0% skip: 1.6%
x264 [info]: mb B I16..4: 1.9% 29.3% 4.6% B16..8: 28.8% 3.6% 5.0% direct: 8.8% skip:17.9% L0:39.9% L1:41.6% BI:18.5%
x264 [info]: 8x8 transform intra:74.3% inter:66.2%
x264 [info]: coded y,uvDC,uvAC intra:93.5% 96.8% 78.6% inter:58.6% 71.7% 22.5%
x264 [info]: ref P L0 58.1% 24.8% 17.1%
x264 [info]: ref B L0 76.7% 23.3%
x264 [info]: kb/s:40000.5

encoded 419 frames, 9.09 fps, 40001.03 kb/sSo far this has happened 9 times out of 9.
The numbers are always different, but it always happens around frame 290.

[Dark Shikari]

x264 [warning]: specified frame type is not compatible with max B-frames
x264 [error]: slice=P but 2pass stats say BI think you have bigger problems than VBV...

[Revgen]

Settings used:

Ran twice, no VBV issues.

Oh well. <shrugs>

I doubt this will ever be a problem on a real movie anyway.

[Dark Shikari]

Oh well. <shrugs>

I doubt this will ever be a problem on a real movie anyway.Sure it will... because I just replicated it with the settings above (12 threads).

2-pass VBV clearly needs work.

[Revgen]

^ So now you tell me. :)

If this Island trailer continues to give you trouble, you can just blame Ben. It's his fault anyway. ;)

[ajp_anton]

I think you have bigger problems than VBV...Sorry that was probably caused by me aborting it. So i re-ran it and edited the post hoping to do so before you read it but I was too slow =)

[Chevy787]

I'm getting a "unsupported input format (DIB )" error when running scripts through the 64 bit version of r1232 x264. If I run the same .avs through the 32 bit version, no errors appear.
Here is the script that is not working.

AVISource("N:\rag\sora\7\filter.avi")
FreezeFrame(15422,15422,15423)
Textsub("N:\rag\sora\7\miles\LogoEP7.ass")
Textsub("N:\rag\sora\7\miles\Sora_no_Mani_Mani_OP_EP7[kana-timed-v2].ass")
Textsub("N:\rag\sora\7\miles\Sora_no_Manimani_ED_EP7[kana-timed].ass")
ConverttoYV12()

I tried a slightly edited script without textsub, and this enabled the 64bit to run.

AVISource("N:\rag\sora\7\filter.avi")
FreezeFrame(15422,15422,15423)
ConverttoYV12()

After further testing, I found Loadplugin() and Tweak() also caused the same error even without the presence of textsub()

Here is my x264 log (using the the first avs mentioned above)

x264.exe --pass 1 --stats Doverlay.avs.264stat --bitrate 1850 --profile high --bframes 16
--deblock 0:0 --subme 7 --merange 8 --keyint 300 --b-adapt 2 --psy-rd 0.8:0.4 --aq-strength0.6
--no-mbtree --min-keyint 10 --weightb --partitions none --me hex --direct auto
--threads auto --thread-input --fps 23.976 --output NUL Doverlay.avs
avis [error]: unsupported input format (DIB )
x264 [error]: could not open input file 'Doverlay.avs'
----------------
x264.exe --pass 2 --stats Doverlay.avs.264stat --bitrate 1850 --profile high --bframes 16
--deblock 0:0 --subme 10 --merange 24 --keyint 300 --ref 16 --trellis 2 --psy-rd 0.8:0.4
--b-adapt 2 --aq-strength 0.6 --no-mbtree --no-dct-decimate --min-keyint 10 --me tesa
--partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --weightb --mixed-refs --no-fast-pskip
--direct auto --threads auto --thread-input --fps 23.976 --output X264vefour.mkv Doverlay.avs
----------------
Done.
Press any key to continue . . . avis [error]: unsupported input format (DIB )
x264 [error]: could not open input file 'Doverlay.avs'

[LoRd_MuldeR]

I'm getting a "unsupported input format (DIB )" error when running scripts through the 64 bit version of r1232 x264.

Is the 64-Bit version of Avisynth installed on your system ???

[Chevy787]

Is the 64-Bit version of Avisynth installed on your system ???

At one point, but I uninstalled it.

[LoRd_MuldeR]

At one point, but I uninstalled it.

So how do you think 64-Bit x264 should be able to use Avisynth input, when there's no 64-Bit Avisynth installed? :rolleyes:

[Chevy787]

So how do you think 64-Bit x264 should be able to use Avisynth input, when there's no 64-Bit Avisynth installed? :rolleyes:

I didn't think it was possible to install both the 32bit and the 64bit version at the same time....is it?
But yes, you do have quite a good point there.

[LoRd_MuldeR]

I didn't think it was possible to install both the 32bit and the 64bit version at the same time....is it?

Should be possible. The 32-Bit DLL goes to "%WINDIR%\SysWow64", while the 64-Bit DLL goes to "%WINDIR%\System32".

(Yes, the 64-Bit system directory is called System32 on 64-Bit Windows ^^)

But yes, you do have quite a good point there.

There is a way to use 64-Bit x264 with 32-Bit Avisynth, but Avisynth must run in a separate 32-Bit process and the video data must be piped into the 64-Bit x264 process.

I made a tool for that purpose: http://forum.doom9.org/showthread.php?t=144140

[Chevy787]

Thanks a bunch for the help

[Dark Shikari]

^ So now you tell me. :)

If this Island trailer continues to give you trouble, you can just blame Ben. It's his fault anyway. ;)Well the result of this is that it appears to be a very odd situation: 2-pass VBV breaks no matter how perfect the input information is--i.e. even multi-multi-pass doesn't fix the problem. Debugging suggests something very weird must be going on internally, so I'm trying to contact Gabriel to see what he can figure out.

Also, in the meantime, 1-pass works great for CBR ;)

[G_M_C]

I am really impressed at the rate of change in x264. The number of patches wich come out and that improve quality or speed is astonishing.

I do hope that that doesnt cause what i often encounter in my hobby-attempts at programming; The more i add or change, the messier the programm tends to get as a whole. I often end up having to go over the whole program to rewrite the basic structure at some point. Hopefully that's not the case now with x264.

Also in this post: I want to giive many cudo's to all developers, but to DS goes even more praise; Cause he almost seems to be the "spokesperson on all things x264" here on the board.

[juGGaKNot]

--pass 1 --preset veryslow --level 3.2 --bitrate 4500 --ref 5 --bframes 3 --ratetol 2.0 --merange 32 --direct temporal --aq-mode 2 --no-fast-pskip --no-dct-decimate --sar 1:1 --aud

300 frames source, uncompressed, avisynth 2.5.8 :

AVIsource("C:\x264\movie.avi")
ConvertToYV12()
LoadPlugin("C:\x264\bin\autocrop.dll")
AutoCrop(0, 16, 16, threshold=0)

error :

avis [info]: 1184x656 @ 30.00 fps (300 frames)
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Slow
x264 [info]: profile High, level 3.2
avis [info]: 1184x656 @ 30.00 fps (300 frames), eta 0:00:00
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Slow
x264 [error]: ratecontrol_init: can't open stats file
x264 [error]: x264_encoder_open failed

gets to the last frame, stalls, x264 windows error box.

jeebs : stalls for 2-3 minutes after first pass, works, output k
techouse : crashes.

[rack04]

300 frames source, uncompressed, avisynth 2.5.8 :



error :



gets to the last frame, stalls, x264 windows error box.

jeebs : stalls for 2-3 minutes after first pass, works, output k
techouse : crashes.

Can you post the source so I can test?

[juGGaKNot]

i will upload on friday

i got the same error with 1222

seems to be GCC 4.4.1 or march=core2 problem

i have to test all 1232 builds first.

[Mtz]

O have this error using Megui:
Assertion failed: dpb_output_delay < pow( 2, sps->vui.nal_hrd_parameters.i_dpb_output_delay_length ), file encoder/set.c, line 679
Posted also in megui thread (http://forum.doom9.org/showthread.php?p=1370933#post1370933) because I don't know who is giving the error.

enjoy,
Mtz

[Underground78]

I think this error was caused by an old version of the nal-hrd patch. Komisar uses this version of the patch for his builds which are included in Megui, I think he should use the newer version that can be found here (http://forum.doom9.org/showthread.php?t=152127) ...

[horvathd]

Hi!

My problem with x264 started when the new source options were introduced. The problem is, when I try to encode a video with non integer fps to mp4, the result play much faster then it should using Windows Media Player.

The problem doesn't come up:
- with Media Player Classic HC
- when I try to encode a video with integer framerate
- encoding to mkv
- using the --fps option in the command line

I'm not sure it's a bug in x264, but with r1376 there is no problem, but it's appears in builds with and without the new input options.

Please look into it. Thanks.

[kieranrk]

I'm not sure it's a bug in x264, but with r1376 there is no problem, but it's appears in builds with and without the new input options.


Can you upload a sample of such a file that doesn't play right?

[horvathd]

Here is a compilation of different versions: http://www.mediafire.com/?ythjzznkd25

The problem is with x264bug_23.976-video.mp4 is I use Windows Media Player.
Now I tried play them with mplayer, and I got the following error with all mp4 files, except the one made with r1376 (x264bug_23.976_1376_-video.mp4): [h264 @ 00B2DC90]reference picture missing during reorder

[VFR maniac]

Hi, horvathd.
Did you install Haali Media Splitter?
If you did this, I think probably that problem caused from Haali's splitter bug.

From rev1379, I implmented EditBox/edts writer for eliminating initial delay from mp4 splitter side.
Haali's splitter has a bug about EditBox, because he misunderstanded timescale of media_time. (MP4Box still misunderstands about this in import function side.)
MPC-HC's internal splitter doesn't have the same bug because this splitter cannot deal with EditBox.

Here is test build from Haali.
http://haali.su/mkv/mkx.e.7.exe
This build had been fixed about media_time issue.
Instead of this correction he broke blank clip function but this is not problem here.
Problem causes only when giving plus delay to mp4.

[horvathd]

Thanks!

With the new Haali Media Splitter there is no problem using Windows Media Player.
But with mplayer the problem is still exists (as it doesn't use the new splitter), so probably they splitter is broken too.

Since it's a problem on the decoding site, to resolve it everybody has to update the splitter but there is no stable build with the fix. Is there any better option for making a video which can be played back without problem than encoding to raw h264 and muxing to mp4 with an external program?

[Dracaena]

Hello there x264 gurus

I had not been using x264 for a number of months, and recently started using it again, through meGUI.
meGUI reports that the version I have is "1376 - Jeeb's patched build v2". The CLI itself reports "x264 core:80 r1376M 3feaec2".

The problem I am having is that x264.exe throws an exception, just at the moment before it would normally return me to the command prompt. Here is a screenshot of the point at which thiss happens:

http://www.users.on.net/~drallim/pic/x264_error.png

The error only occurs when outputting to .mp4, however the resulting file is perfectly usable and does not appear to contain any errors. Outputting to .mkv, .flv, or .264 works normally.
I have tried:

Using a very simple command line (with only output and input specified)
using very simple avs script as input
breaking the process before encoding is finished (ctrl+c)
various different switches - eg. disabling CPU optimisations, quiet mode etc.

In all cases the error occurs right before it would normally report "encoded x frames, x.x fps, x.x kb/s".
In all cases the resulting .mp4 file was usable and did not appear corrupt in any way.

Needless to say I will be using one of the other output formats until this is fixed because it prevents meGUI from proceeding to the next job in the queue. Anyway I thought it might be helpful to make somebody aware of it.

[nurbs]

Try replacing your x264.exe with a current build. Maybe it was fixed somewhere along the way or the issue is only with this build for whatever reason. None of the options you use have changed since r1376 so there shouldn't be any problems.

Another thing: --b-pyramid strict is only needed for blu-ray and you are obviously not encoding blu-ray. --b-pyramid normal is set by default in recent builds.

[Dracaena]

Ah. I had assumed that meGUI would have downloaded the most recent build.
I went and got rev1471 from x264.nl, and the error does not occur.

[nurbs]

The reason for that is that some defaults have recently changed in x264 and also the parameters the NAL-HRD patch takes have changed and those changes haven't been integrated into MeGUI yet. With the settings you use it doesn't make a difference, but with (like the old --nal-hrd option) it would lead to x264 giving an error massage. I guess they'll push a new one out with the next stable release.

[ajp_anton]

Isn't rc-lookahead also used for something with VBV even though mb-tree is off? Because when I do turn mb-tree off, x264 will always write "rc_lookahead=0" to the stream.

[b66pak]

hi,

there is a bug in the resize filter (tested with "x264 core:108 r1788 c764e29", "x264 core:107 r1772 c9dad9e"):

source: 704×528 (DAR 4:3, SAR 1:1)

destination: 480×270 (DAR 16:9, SAR 1:1)

my line:

"--vf resize:width=480,height=270,fittobox=both"

result:

lavf [info]: 704x528p 1:1 @ 25/1 fps (vfr)
resize [info]: resizing to 358x268
x264 [info]: using SAR=1/1

it should be:

resizing to 360×270

best regards…
_

P.S. last bug free build: "x264 core:107 r1766 f9f0035"

[kemuri-_9]

you had stated elsewhere/previously that the issue was only being exhibited in certain builds, (iirc it was only JEEBs builds)...
is the issue exhibiting itself in jarod's x264.nl builds (the x86 ones) as well now?

[b66pak]

yes...i am talking about x264.nl builds...
_

[kemuri-_9]

I looked into this previously when J_Darnley was poking me about it in between my battling with viruses on my pc.

the issue appears to be how gcc is optimizing the rounding routine used in the resize filter.
once i told gcc that it couldn't inline the function, the circumstances you provide give the expected 360x270 resolution rather than the unexpected 358x268 resolution.

i thought i had told J_Darnley this to have him confirm it some several days ago, but i never heard on this...

[Dark Shikari]

"Optimizing" does not affect output. Miscompilation does. If it's miscompiling, you should investigate what the difference is.

Just throwing NOINLINEs at the problem is not a solution.

[J_Darnley]

Why not just replace the problem code with the standard math functions of round() and trunc()?

[EDIT] http://pastebin.com/BUxKr4h2 This was just applied, I'm linking it so that anyone can use it now, if they need or wish to.

[b66pak]

if anyone need a vf resize bug free...Komisar (http://komisar.gin.by/) builds are OK...
_

[Maccara]

Why not just replace the problem code with the standard math functions of round() and trunc()?

Is round() guaranteed to have the same implementation on every system nowadays? I know it didn't use to.

[b66pak]

new set of tests with "x264 core:108 r1790 8eaf8a6" (x264.nl build)


line:
--vf resize:width=720,height=480,sar=32:27,fittobox=both

result:
lavf [info]: 720x720p 1:1 @ 24/1 fps (vfr)
resize [info]: resizing to 404x480
x264 [info]: using SAR=32/27

lavf [info]: 960x720p 1:1 @ 24/1 fps (vfr)
resize [info]: resizing to 540x480
x264 [info]: using SAR=32/27

lavf [info]: 1120x720p 1:1 @ 24/1 fps (vfr)
resize [info]: resizing to 606x462 it should be 630x480
x264 [info]: using SAR=32/27

lavf [info]: 1280x720p 1:1 @ 24/1 fps (vfr)
resize [info]: resizing to 606x404 it should be 720x480
x264 [info]: using SAR=32/27

lavf [info]: 1280x640p 1:1 @ 24/1 fps (vfr)
resize [info]: resizing to 606x360 it should be 720x426
x264 [info]: using SAR=32/27


line:
--vf resize:width=720,height=480,sar=8:9,fittobox=both

result:
lavf [info]: 720x720p 1:1 @ 24/1 fps (vfr)
resize [info]: resizing to 480x426 it should be 540x480
x264 [info]: using SAR=8/9

lavf [info]: 960x720p 1:1 @ 24/1 fps (vfr)
resize [info]: resizing to 640x426 it should be 720x480
x264 [info]: using SAR=8/9

lavf [info]: 1120x720p 1:1 @ 24/1 fps (vfr)
resize [info]: resizing to 720x410
x264 [info]: using SAR=8/9

lavf [info]: 1280x720p 1:1 @ 24/1 fps (vfr)
resize [info]: resizing to 720x360
x264 [info]: using SAR=8/9

lavf [info]: 1280x640p 1:1 @ 24/1 fps (vfr)
resize [info]: resizing to 720x320
x264 [info]: using SAR=8/9


line:
--vf resize:width=720,height=576,sar=64:45,fittobox=both

result:
lavf [info]: 720x720p 1:1 @ 24/1 fps (vfr)
resize [info]: resizing to 404x576
x264 [info]: using SAR=64/45

lavf [info]: 960x720p 1:1 @ 24/1 fps (vfr)
resize [info]: resizing to 506x540 it should be 540x576
x264 [info]: using SAR=64/45

lavf [info]: 1120x720p 1:1 @ 24/1 fps (vfr)
resize [info]: resizing to 506x462 it should be 630x576
x264 [info]: using SAR=64/45

lavf [info]: 1280x720p 1:1 @ 24/1 fps (vfr)
resize [info]: resizing to 506x404 it should be 720x576
x264 [info]: using SAR=64/45

lavf [info]: 1280x640p 1:1 @ 24/1 fps (vfr)
resize [info]: resizing to 506x360 it should be 720x512
x264 [info]: using SAR=64/45


line:
--vf resize:width=720,height=576,sar=16:15,fittobox=both

result:
lavf [info]: 720x720p 1:1 @ 24/1 fps (vfr)
resize [info]: resizing to 540x576
x264 [info]: using SAR=16/15

lavf [info]: 960x720p 1:1 @ 24/1 fps (vfr)
resize [info]: resizing to 674x540 it should be 720x576
x264 [info]: using SAR=16/15

lavf [info]: 1120x720p 1:1 @ 24/1 fps (vfr)
resize [info]: resizing to 674x462 it should be 720x494
x264 [info]: using SAR=16/15

lavf [info]: 1280x720p 1:1 @ 24/1 fps (vfr)
resize [info]: resizing to 674x404 it should be 720x432
x264 [info]: using SAR=16/15

lavf [info]: 1280x640p 1:1 @ 24/1 fps (vfr)
resize [info]: resizing to 674x360 it should be 720x384
x264 [info]: using SAR=16/15
_

[b66pak]

rounding errors still exists with "x264 core:108 r1790 8eaf8a6" and "x264 core:110 r1804 e89c4cf" (both x264.nl builds)

line:
--vf resize:width=720,height=528,sar=10:11,fittobox=both

result:
lavf [info]: 960x720p 1:1 @ 24/1 fps (vfr)
resize [info]: resizing to 704x478 it should be 704x480
x264 [info]: using SAR=10/11

_

[J_Darnley]

Why don't you guys try this: http://pastebin.com/wckZhxbr

And before you report any rounding errors, run the decimal math yourself.

[MasterNobody]

Why don't you guys try this: http://pastebin.com/wckZhxbr

And before you report any rounding errors, run the decimal math yourself.
Your variant will run out of the box with such params:
--vf resize:width=721,height=528,sar=10:11,fittobox=both --input-res 960x720
It will try to resize to 722x492

Here is my current patch: http://privatepaste.com/afda985c84 (also have cosmetics to error messages because "unsetxunset" looks ugly)

[J_Darnley]

>width=721
Now that sounds like a reason for an error message

[b66pak]

--vf resize:width=720,height=528,sar=10:11,fittobox=both --input-res 960x720

@MasterNobody with your patch i get 720x490 (sar=10:11)...players like MPC-HC or VLC display this like 720x538(9) (sar=1:1) witch defeat the conditions:

width=720,height=528,sar=10:11,fittobox=both

the correct result should be 704x480 (sar=10:11) displayed as 704x528 (sar=1:1)...
_

L.E. on a second look:

i was using:

--vf resize:width=720,height=528,sar=10:11,fittobox=both/pad:width=720,height=480 --input-res ANY

as a workaround for this (see up (http://forum.doom9.org/showthread.php?p=1460256#post1460256)):

--vf resize:width=720,height=480,sar=10:11,fittobox=both/pad:width=720,height=480 --input-res ANY

your patch fix it! so i will use it...

thanks a lot!
_

[MasterNobody]

--vf resize:width=72o,height=528,sar=10:11,fittobox=both --input-res 960x720

@MasterNobody with your patch i get 720x490 (sar=10:11)...players like MPC-HC or VLC display this like 720x538(9) (sar=1:1) witch defeat the conditions:

width=721,height=528,sar=10:11,fittobox=both

the correct result should be 704x480 (sar=10:11) displayed as 704x528 (sar=1:1)...
_
Why 704x528? Box limits coded resolution not the displaying resolution (the method of scaling in decoder can be any so we can't suppose anyone). So 720x490 is fully inside the limits of 720x528.

[b66pak]

hi,

i edited my post...see the L.E...actually the current implementation for fittobox is based on displaying resolution (no matter how you apply the sar the final result is in the box)! take look at my examples (http://forum.doom9.org/showthread.php?p=1460256#post1460256)!

704x480 (sar=10:11) can be displayed as 704x528 (sar=1:1) or 640x480 (sar=1:1) both in the box limits...
_

[MasterNobody]

hi,

i edited my post...see the L.E...actually the current implementation for fittobox is based on displaying resolution! take look at my examples (http://forum.doom9.org/showthread.php?p=1460256#post1460256)!
_
And this is incorrect because most of devices limits coded resolution not displaying IMHO. For displaying they can shrink instead of enlarge.

[b66pak]

this is my opinion too...sar should be applied only horizontal (like in tv sets or dvd players)...i am not defending the current implementation for fittobox...i explain it...
_

[J_Darnley]

fittobox works exactly as it should. You say "I need this resolution and sar" and that's what you get.

[rasmusb]

This is my first bug report for x264 so while I've tried my best to be precise and concise bear with me if the problem is on my end or I'm missing something obvious. That said, I sometimes experience a corrupted blocky or gray output from x264. From my experiments the problems occur when using --preset slow but goes away with --preset medium. I have found that "--preset fast --b-adapt 2 --rc-lookahead 50" is enough to provoke the error. I have isolated 150 frames (about 27MB) from the source which by itself will show the error at about frame 128. I have used both a recent ffdshow and dxva from MPC-HC to decode the stream with the same result. I have also tried encoding it on another computer with a fresh Windows 7 install with only Haali Splitter, ffdshow and AviSynth installed with the same result.

I have tested the following builds (all x86):
(from x264.nl) x264 core:112 r1834 a51816a - The bitstream freezes for about 15 frames and then the frame gets gray
(from komisar) x264 core:110 r1820 fdcf2ae - Same as above
(from x264.nl) x264 core:83 r1391 3d0f110 - The bitstream gets blocky for about 15 frames

The source stream can be found here: http://www.megaupload.com/?d=MCL8OF16


Example images:
http://i53.tinypic.com/2vajn08.jpg
x264.r1391 --b-adapt 2 --rc-lookahead 50

http://i56.tinypic.com/2u4j0cx.jpg
x264.r1834 --b-adapt 2 --rc-lookahead 50

http://i52.tinypic.com/21d0nl5.jpg
x264.r1834 --preset medium



The AviSynth script:
DirectShowSource("new.split.1.m2ts", fps=23.976, audio=false, convertfps=true)

Log for x264.nl r1834:
x264 --b-adapt 2 --rc-lookahead 50 -o out.mp4 00009_demux.avs

avs [info]: 1920x1080p 0:0 @ 2500000/104271 fps (cfr)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile High, level 4.0
x264 [info]: frame I:2 Avg QP:21.07 size: 97845
x264 [info]: frame P:58 Avg QP:21.22 size: 30003
x264 [info]: frame B:92 Avg QP:26.31 size: 17763
x264 [info]: consecutive B-frames: 16.0% 1.3% 16.0% 66.7%
x264 [info]: mb I I16..4: 24.9% 55.8% 19.3%
x264 [info]: mb P I16..4: 1.8% 5.5% 1.4% P16..4: 20.1% 6.1% 3.5% 0.0% 0.0% skip:61.6%
x264 [info]: mb B I16..4: 0.6% 1.6% 0.5% B16..8: 39.5% 4.7% 1.2% direct: 2.2% skip:49.8% L0:42.4% L1:51.2% BI: 6.3%
x264 [info]: 8x8 transform intra:61.1% inter:73.1%
x264 [info]: coded y,uvDC,uvAC intra: 65.4% 31.2% 5.0% inter: 12.1% 4.2% 0.0%
x264 [info]: i16 v,h,dc,p: 47% 22% 11% 20%
x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 16% 12% 18% 6% 11% 11% 10% 7% 8%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 16% 14% 18% 6% 12% 11% 10% 6% 7%
x264 [info]: i8c dc,h,v,p: 65% 16% 17% 2%
x264 [info]: Weighted P-Frames: Y:0.0% UV:0.0%
x264 [info]: ref P L0: 59.4% 14.2% 19.2% 7.1%
x264 [info]: ref B L0: 90.3% 7.7% 2.0%
x264 [info]: ref B L1: 96.1% 3.9%
x264 [info]: kb/s:4505.08

encoded 152 frames, 5.64 fps, 4505.94 kb/s

[JEEB]

Do not confirm.

Got the sample, used my 1834 patched build as well as the 64bit pure build that gets put on x264.nl, read with x264's ffms input as the source is progressive H.264. Output was practically the same.

x264 new.split.1.m2ts --b-adapt 2 --rc-lookahead 50 -o out.mp4
ffms [info]: 1920x1080p 1:1 @ 24000/1001 fps (vfr)
mp4 [info]: audio muxing feature is disabled.
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 FastShuffle SSE4.1 Cache64
x264 [info]: profile High, level 4.0
x264 [info]: frame I:2 Avg QP:21.12 size: 97010
x264 [info]: frame P:42 Avg QP:23.46 size: 44139
x264 [info]: frame B:109 Avg QP:26.32 size: 16754
x264 [info]: consecutive B-frames: 0.7% 1.3% 23.8% 74.2%
x264 [info]: mb I I16..4: 25.5% 55.9% 18.7%
x264 [info]: mb P I16..4: 2.7% 7.5% 1.8% P16..4: 33.9% 10.2% 5.9% 0.0% 0.0% skip:37.9%
x264 [info]: mb B I16..4: 0.2% 1.0% 0.3% B16..8: 40.2% 4.7% 1.2% direct: 2.1% skip:50.3% L0:42.6% L1:51.5% BI: 5.9%
x264 [info]: 8x8 transform intra:61.8% inter:72.7%
x264 [info]: coded y,uvDC,uvAC intra: 66.0% 30.5% 4.0% inter: 14.2% 4.5% 0.0%
x264 [info]: i16 v,h,dc,p: 45% 25% 10% 20%
x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 16% 12% 16% 6% 12% 12% 10% 8% 8%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 16% 14% 17% 6% 13% 12% 10% 6% 7%
x264 [info]: i8c dc,h,v,p: 64% 17% 17% 2%
x264 [info]: Weighted P-Frames: Y:0.0% UV:0.0%
x264 [info]: ref P L0: 57.9% 14.9% 19.2% 8.0%
x264 [info]: ref B L0: 89.0% 8.7% 2.3%
x264 [info]: ref B L1: 94.8% 5.2%
x264 [info]: kb/s:4856.70

remux [100.00%], 3787/3787 KiB, 378722 KiB/s, total elapsed 0:00:00

encoded 153 frames, 3.47 fps, 4857.59 kb/s


Output file: clicky (http://www.mediafire.com/?fi6jxei4jabobwg)

Seems fine as far as I'm concerned, tested with both a libav*-based decoder as well as CoreAVC's VPx-based decoder. Doesn't seem to be a bug with x264, but with your input (It could also be the playback setup you use to play the output, but I'm betting on the first option as you seem to be using DirectShowSource).

I would like to mention here, that although DirectShowSource _does_ come with Avisynth, it surely isn't the best way of loading your sources. It isn't frame-exact in any way and is prone to whatever errors you might have in your DirectShow setup (as it uses DirectShow filter chains for input). Please try another way of input, f.ex. x264's ffms input, the ffmpegsource Avisynth plugin, or even the DSS2 plugin by Haali, which is pretty good on things that Haali's splitter splits as long as you have your DirectShow environment in a good condition and Haali's splitters installed for the container format you want to load (although I had visual corruption with DSS2 on Haali's splitters >=2010/05/20). Not to mention the H.264-related tools by Neuron2.

For MPEG-2 sources there are other plugins to load such streams via d2v files.

[rasmusb]

Thanks for your quick reply!

The problem goes away when using the .m2ts directly as you did (is this ffms input?). The problem is still there when I tried dss2. I use a recent Haali splitter, definately >=2010/05/20.

The reason I've been using DirectShowSource is that I've been using MeGUI and that's how it works. I'll do the video encoding from the command line until I find a way that works from the GUI.

Thanks again for your help!

[nurbs]

MeGUI comes with ffmpegsource (ffms). Use the file indexer and load the index that it produces in the Avisynth script generator.

[b66pak]

can you please investigate the HUGE timescale/timebase problem described here (http://forum.doom9.org/showthread.php?p=1479455#post1479455)?
_

[kemuri-_9]

can you please investigate the HUGE timescale/timebase problem described here (http://forum.doom9.org/showthread.php?p=1479455#post1479455)?
_

post your sample with the retarded timebase.

[b66pak]

here (http://www.mediafire.com/?dqr1ff7abcj8b8j) is a small sample...
_

[magic144]

Just been looking at an issue over at Donald Graft's forum...

There is a weird stuttering issue when trying to encode a particular clip from a BBC disc, using a script that involves DGSource() and srestore()...

I've uploaded all the relevent files for this issue here:-

http://www.mediafire.com/?e8s6eqse62n8u


It seems as if x264 is dropping some frames somehow?
If I load the (METHOD 2) test.avs script into VirtualDub, I can generate an .avi (HuffyYUV) with perfectly smooth playback.
I have also since discovered that if I use x264vfw from within VirtualDub, I can produce an .avi (h.264) with perfectly smooth playback (BUT ONLY if VirtualDub Hack is selected).

Here is the info on my initial experiment/findings describing the stuttering issue...


----

Source video clip is opening 30s of episode, video30s.mkv.
Index clip using DGIndexNV => video30s.dgi

2 approaches to encoding same clip:
METHOD 1. Convert via Huffyuv intermediate.

Convert the clip into a Huffyuv .avi using this script,
---huffy.avs---
LoadPlugin("C:\Users\User\Desktop\dgdecnv2038\DGDecodeNV.dll")
DGSource("video30s.dgi")
TRIM(0,150)
---
This produces (via VirtualDub using ffdshow/HuffYUV) video30s-huffy150fr.avi.

Now use a second script to encode to h.264.
NB the video30s-huffy150fr.grf file is produced using MONOGRAM GraphStudio (v0.3.2.0 Beta), just rendering video30s-huffy150fr.avi via FileSource (Async.), AVI Splitter and ffdshow (rev3721).
---testhuffy.avs---
Import("Srestore.avsi")
LoadPlugin("mt_masktools-26.dll")
LoadCplugin("C:\Program Files\MeGUI\tools\yadif\yadif.dll")
DirectShowSource("video30s-huffy150fr.grf",audio=false)
AssumeFPS(30000,1001) # as per source
yadif(mode=1,order=1)
srestore(frate=25,speed=-1)
---
This produces testhuffy-muxed.mkv, which plays back smoothly.


METHOD 2. Use this more straightforward script.
---test.avs---
LoadPlugin("C:\Users\User\Desktop\dgdecnv2038\DGDecodeNV.dll")
Import("Srestore.avsi")
LoadPlugin("mt_masktools-26.dll")
LoadCplugin("C:\Program Files\MeGUI\tools\yadif\yadif.dll")
DGSource("video30s.dgi")
yadif(mode=1,order=1)
srestore(frate=25,speed=-1)
TRIM(0,125) #125 is 150*5/6
---
This produces (via MeGUI) test-muxed.mkv. As you can see, there is a noticeable temporal stutter at the start of playback.


The question is, what induces the initial stuttering in approach 2. This is the preferred approach since it doesn't involve a very large intermediate file! Something, however, is not "playing nice".


FYI...

Command lines MeGUI generates when encoding the clips:-
----
"C:\Program Files\MeGUI\tools\x264\x264.exe" --level 4.1 --preset slow --crf 20 --no-dct-decimate --sar 1:1 --output "D:\Temp\_org\Stutter\testhuffy.264" "D:\Temp\_org\Stutter\testhuffy.avs"
"C:\Program Files\MeGUI\tools\x264\x264.exe" --level 4.1 --preset slow --crf 20 --no-dct-decimate --sar 1:1 --output "D:\Temp\_org\Stutter\test.264" "D:\Temp\_org\Stutter\test.avs"

Srestore() is found here:-
http://avisynth.org/mediawiki/Srestore
also requires Masktools2, found here:-
http://avisynth.org/mediawiki/MaskTools2



MASSIVE thanks if anybody can pinpoint what is going wrong here!
I'd like to use x264.exe CLI as I have always used it in the past without issue...

[kemuri-_9]

here (http://www.mediafire.com/?dqr1ff7abcj8b8j) is a small sample...
_

with an altered version of x264cli to get more output:


$ ./x264 -o ffms.mkv "F:\Download\sample\sample.mkv" --tcfile-out ffms_tc.txt && mv pts.txt ffms_raw_pts.txt
ffms [info]: 1280x532p 1:1 @ 23999/1001 fps (vfr)
ffms [info]: timebase = 1/1000000000



# timecode format v2
0.000000
42.000000
84.000000
125.000000
167.000000
209.000000



0
42000000
84000000
125000000
167000000
209000000



$ ./x264 -o lavf.mkv "F:\Download\sample\sample.mkv" --demuxer lavf --tcfile-out lavf_tc.txt && mv pts.txt lavf_raw_pts.txt
[matroska,webm @ 003fc330] Estimating duration from bitrate, this may be inaccurate
lavf [info]: 1280x532p 0:1 @ 24000/1001 fps (vfr)
lavf [info]: timebase = 1/1000



# timecode format v2
0.000000
42.000000
84.000000
125.000000
167.000000
209.000000



0
42
84
125
167
209


Overall lesson of the story (or simply TL;DR):
ffms outputs PTS that are 1,000,000x larger than lavf, so it needs a 1,000,000x larger timebase denominator.
(1000 of the final 1billion is x264cli compensating for ffms's large PTS output values)

though this behavior is only exclusive to mkvs with ffms's internal demuxer, as ffms has it's own mkv demuxer separate of lavf's.

There was recently some API added to ffms to support forcing the demuxer, but it's not used by x264cli, so ffms defaults to its internal mkv demuxer here.

[b66pak]

thank you for the explanation...
_

[b66pak]

it looks like lavf demuxer can deliver HUGE timescale/timebase too! (for .avi)...

example one:

mediainfo:
Format : AVI
Format/Info : Audio Video Interleave
File size : 349 MiB
Duration : 44mn 2s
Overall bit rate : 1 108 Kbps
Writing application : VirtualDubMod 1.5.10.2 (build 2540/release)
Writing library : VirtualDubMod build 2540/release

Video
Format : MPEG-4 Visual
Format profile : Advanced Simple@L5
Format settings, BVOP : 1
Format settings, QPel : No
Format settings, GMC : No warppoints
Format settings, Matrix : Default (H.263)
Muxing mode : Packed bitstream
Codec ID : XVID
Codec ID/Hint : XviD
Duration : 44mn 2s
Bit rate : 966 Kbps
Width : 640 pixels
Height : 352 pixels
Display aspect ratio : 16:9
Frame rate : 23.976 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.179
Stream size : 304 MiB (87%)
Writing library : XviD 1.2.1 (UTC 2008-12-04)

ffmpeg info:
Seems stream 0 codec frame rate differs from container frame rate: 23.98 (65535/2733) -> 23.98 (10000000/417083)

x264 lavf info:
lavf [info]: 640x352p 1:1 @ 10000000/417083 fps (vfr)

resulting mp4 (x264 gpac muxer) mp4box info:
Track # 1 Info - TrackID 1 - TimeScale 10000000 - Duration 00:00:41.708

_

[b66pak]

example two:

mediainfo:
Format : AVI
Format/Info : Audio Video Interleave
File size : 350 MiB
Duration : 42mn 32s
Overall bit rate : 1 149 Kbps
Writing application : transcode-1.0.6

Video
IFormat : MPEG-4 Visual
Format profile : Advanced Simple@L5
Format settings, BVOP : 2
Format settings, QPel : No
Format settings, GMC : No warppoints
Format settings, Matrix : Default (H.263)
Codec ID : XVID
Codec ID/Hint : XviD
Duration : 42mn 32s
Bit rate : 991 Kbps
Width : 624 pixels
Height : 352 pixels
Display aspect ratio : 16:9
Frame rate : 23.976 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.188
Stream size : 301 MiB (86%)
Writing library : XviD 1.2.1 (UTC 2008-12-04)

ffmpeg info:
Seems stream 0 codec frame rate differs from container frame rate: 23.98 (65535/2733) -> 23.98 (2997003/125000)

x264 lavf info:
lavf [info]: 624x352p 1:1 @ 2997003/125000 fps (vfr)

resulting mp4 (x264 gpac muxer) mp4box info:
Track # 1 Info - TrackID 1 - TimeScale 2997003 - Duration 00:00:41.708
_

[b66pak]

example three:

mediainfo:
Format : AVI
Format/Info : Audio Video Interleave
File size : 350 MiB
Duration : 41mn 53s
Overall bit rate : 1 168 Kbps
Writing application : transcode-1.0.4

Video
Format : MPEG-4 Visual
Format profile : Advanced Simple@L5
Format settings, BVOP : 2
Format settings, QPel : No
Format settings, GMC : No warppoints
Format settings, Matrix : Default (H.263)
Codec ID : XVID
Codec ID/Hint : XviD
Duration : 41mn 53s
Bit rate : 999 Kbps
Width : 624 pixels
Height : 352 pixels
Display aspect ratio : 16:9
Frame rate : 23.976 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.190
Stream size : 299 MiB (86%)
Writing library : XviD 1.2.1 (UTC 2008-12-04)

ffmpeg info:
Seems stream 0 codec frame rate differs from container frame rate: 23.98 (24000/1001) -> 23.98 (2997003/125000)

x264 lavf info:
lavf [info]: 624x352p 1:1 @ 24000/1001 fps (vfr)

resulting mp4 (x264 gpac muxer) mp4box info:
Track # 1 Info - TrackID 1 - TimeScale 2997003 - Duration 00:00:41.708
_

[b66pak]

it will be nice o see timebase info in the vanilla builds too!

ffms [info]: timebase = 1/1000000000

lavf [info]: timebase = 1/1000
_

[kemuri-_9]

all these timebase issues you keep coming with seem to be arising from the fact that you are using files that are likely encoded from avisynth where directshowsource was used (it causes very VERY terrible fps without AssumeFPS usage)

use --fps on these .avi files to correct the framerate, they're not vfr anyway.

[b66pak]

how about lavf & ffms (or maybe x264) forcing timebase to 1/1000 (or 1/10000 if more precision is needed - my personal opinion is that miliseconds (1/1000) are enough) when HUGE timescale/timebase are detected?
_

[b66pak]

or even better, like this:

new_timebase = 1000/[int(1000/huge_timebase)]

for example one:

new_timebase = 1000/[int(1000/(417083/10000000)] = 1000/23976 = 125/2997

for example two & three:

new_timebase = 1000/[int(1000/(125000/2997003)] = 1000/23976 = 125/2997
_

[kemuri-_9]

it should be the user's job to handle quirks such as these in their input.

also, these avis are bad examples to prove a point with, use --fps on them.

[Juce]

x264 crashes with this:
x264 -q 0 -o video.mp4 checkerboard1280x720.avs

This works:
x264 -q 0 -o video.mkv checkerboard1280x720.avs

checkerboard1280x720.avs
ImageSource("checkerboard1280x720.png", 0, 49, 25)

checkerboard1280x720.png (http://img193.imageshack.us/img193/3010/checkerboard1280x720.png)

[LoRd_MuldeR]

x264 crashes with this:
x264 -q 0 -o video.mp4 checkerboard1280x720.avs

This works:
x264 -q 0 -o video.mkv checkerboard1280x720.avs

Might be related or not:
https://github.com/DarkShikari/x264-devel/commit/a073bb87aa56dab04829bb470d6e8dad7a6112e2

[kemuri-_9]

Might be related or not:
https://github.com/DarkShikari/x264-devel/commit/a073bb87aa56dab04829bb470d6e8dad7a6112e2

yes, that commit is to fix the reported issue.

[Juce]

Something wrong with dithering and/or colorspace conversion.

Encoding without dithering:
x264 rgb48.raw -o video1.mkv --demuxer raw --input-csp rgb --input-depth 16 --input-res 1280x720 --fps 25 --preset slow -q 0 --vf resize:,,,,yv12:8
raw [info]: 1280x720p 0:0 @ 25/1 fps (cfr)
resize [warning]: converting from rgb48le to yuv420p
x264 [info]: using cpu capabilities: MMX2 SSE2Slow SlowCTZ
x264 [info]: profile High 4:4:4 Predictive, level 3.1, bit depth 8
x264 [info]: frame I:1 Avg QP: 0.00 size: 30762
x264 [info]: frame P:9 Avg QP: 0.00 size: 21888
x264 [info]: mb I I16..4: 98.5% 1.5% 0.0%
x264 [info]: mb P I16..4: 23.6% 1.4% 0.0% P16..4: 59.2% 1.3% 0.2% 0.0% 0
.0% skip:14.2%
x264 [info]: 8x8 transform intra:4.4% inter:89.1%
x264 [info]: coded y,uvDC,uvAC intra: 66.4% 72.2% 71.2% inter: 27.7% 64.7% 63.7%

x264 [info]: i16 v,h,dc,p: 56% 43% 1% 0%
x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 5% 7% 52% 11% 8% 3% 5% 5% 3%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 0% 0% 100% 0% 0% 0% 0% 0% 0%

x264 [info]: i8c dc,h,v,p: 32% 43% 25% 0%
x264 [info]: Weighted P-Frames: Y:0.0% UV:0.0%
x264 [info]: ref P L0: 96.2% 0.0% 3.6% 0.2% 0.0% 0.0%
x264 [info]: kb/s:4555.10

encoded 10 frames, 4.57 fps, 4565.54 kb/s

Encoding with dithering:
x264 rgb48.raw -o video2.mkv --demuxer raw --input-csp rgb --input-depth 16 --input-res 1280x720 --fps 25 --preset slow -q 0
raw [info]: 1280x720p 0:0 @ 25/1 fps (cfr)
resize [warning]: converting from rgb48le to yuv420p16le
x264 [info]: using cpu capabilities: MMX2 SSE2Slow SlowCTZ
x264 [info]: profile High 4:4:4 Predictive, level 3.1, bit depth 8
x264 [info]: frame I:1 Avg QP: 0.00 size: 68428
x264 [info]: frame P:9 Avg QP: 0.00 size: 53652
x264 [info]: mb I I16..4: 98.4% 1.5% 0.0%
x264 [info]: mb P I16..4: 10.6% 1.0% 0.0% P16..4: 79.7% 3.4% 2.3% 0.0% 0
.0% skip: 3.0%
x264 [info]: 8x8 transform intra:5.0% inter:86.2%
x264 [info]: coded y,uvDC,uvAC intra: 64.5% 92.5% 92.3% inter: 28.2% 94.8% 94.6%

x264 [info]: i16 v,h,dc,p: 60% 39% 1% 0%
x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 4% 7% 54% 12% 7% 3% 6% 5% 3%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 0% 0% 100% 0% 0% 0% 0% 0% 0%

x264 [info]: i8c dc,h,v,p: 73% 8% 14% 5%
x264 [info]: Weighted P-Frames: Y:0.0% UV:0.0%
x264 [info]: ref P L0: 87.6% 0.0% 8.7% 2.1% 1.0% 0.5%
x264 [info]: kb/s:11025.84

encoded 10 frames, 3.25 fps, 11036.28 kb/s

The bitrate increases but banding looks the same.

without dithering: http://img39.imageshack.us/img39/5747/video1i.png
with dithering: http://img198.imageshack.us/img198/811/video2a.png
orginal: http://img688.imageshack.us/img688/6218/frame000.png

rgb48.raw: http://www.mediafire.com/file/hbpe09m0u04v3g5/rgb48.raw

[b66pak]

starting with 1947 build the .mp4 output OOS...tested with:

x264 core:115 r1947 b5a8ad7 - gpac (x264.nl)

x264 0.115.1947+524M_L-SMASH a8e6471 - l-smash (x5-452 build)

x264 core:115 r1947+532 361db68 - l-smash (fushizen build)

x264 core:115 r1947kMod b5a8ad7 - gpac (komisar build)


my line:
x264 --profile main --level 3 --weightp 0 --ref 3 --b-pyramid 0 --no-weightb --vbv-maxrate 10000 --vbv-bufsize 10000 -o video4psp.mp4 video4psp.avs
_

[komisar]

b66pak, my clear build with "old-golgol777" gpac, kMod-build with l-smash. (and you test 32/64-bit?)

[b66pak]

tested the clear build too...the tests are for 32bit builds...i don't think is the muxers (gpac or l-smash) because are both affected...if i demux the resulting .mp4 to raw h264 and remux it again to mp4 the problem is gone...
_

[komisar]

b66pak, is the 1936 version is clear?

[b66pak]

sorry...if found that it is a problem for some media players unable to deal with .mp4 files created without "--dts compress"...adding "--dts compress" to the encoding line fixed the issue...
_

[upyzl]

Excuse me for my offensiveness

I think --merange should be restricted when using --me umh/esa/tesa

now --merange can be set to 2^31-1 (int upper limit), but it easily causes x264.exe crash because of user's RAM(maybe...)

and also, higher merange doesn't mean higher PSNR/SSIM(especially >64), so it really no need to use very high merange, then restrict it should have no side-effect

pic shot:http://i.imgur.com/61WCQ.png

[Juce]

Something wrong with dithering and/or colorspace conversion.I wrote a converter to circumvent the bug:/* rgb48le_to_yuv420p16le.c
Not copyrighted -- provided to the public domain
gcc -O2 -mfpmath=sse -march=native rgb48le_to_yuv420p16le.c -o rgb48le_to_yuv420p16le

http://www.itu.int/rec/R-REC-BT.709-5-200204-I/en page 19 (PDF page 21) */

#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>

/* Rec. 709 */
#define kb 0.0722
#define kr 0.2126

/* Rec. 601 */
//#define kb 0.114
//#define kr 0.299

#define input_depth 16
#define output_depth 16

#if input_depth > 8
#define input_datatype uint16_t
#else
#define input_datatype uint8_t
#endif

#if output_depth > 8
#define output_datatype uint16_t
#else
#define output_datatype uint8_t
#endif

// SET_BINARY_MODE() from http://www.zlib.net/zpipe.c
#if defined(MSDOS) || defined(OS2) || defined(WIN32) || defined(__CYGWIN__)
# include <fcntl.h>
# include <io.h>
# define SET_BINARY_MODE(file) setmode(fileno(file), O_BINARY)
#else
# define SET_BINARY_MODE(file)
#endif


int main(int argc, char **argv) {
SET_BINARY_MODE(stdin);
SET_BINARY_MODE(stdout);

void print_usage() {
fprintf(stderr, "\nusage: %s <width> <height>\n", argv[0]);
fprintf(stderr, " reads RGB48 from stdin\n");
fprintf(stderr, " writes 420p16 Rec. 709 (or something) to stdout\n");
fprintf(stderr, " <width> and <height> must be even\n");
}

void print_end_msg(int frames) {
fprintf(stderr, "\n%s converted %i frames\n", argv[0], frames);
}

int width, height;

if (argc != 3) { print_usage(); return 1; }
if (sscanf(argv[1], "%i", &width) != 1) { print_usage(); return 1; }
if (sscanf(argv[2], "%i", &height) != 1) { print_usage(); return 1; }
if (width % 2 || height % 2 || width < 2 || height < 2) { print_usage(); return 1; }

input_datatype *rgb_buf;
output_datatype *y_buf, *cb_buf, *cr_buf;

rgb_buf = malloc(width * 2 * sizeof(rgb_buf[0]) * 3);
y_buf = malloc(width * 2 * sizeof(y_buf[0]));
cb_buf = malloc((width / 2) * (height / 2) * sizeof(cb_buf[0]));
cr_buf = malloc((width / 2) * (height / 2) * sizeof(cr_buf[0]));

const double ry = kr * 219 * (1 << (output_depth - 8)) / ((1 << input_depth) - 1);
const double gy = (1 - kr - kb) * 219 * (1 << (output_depth - 8)) / ((1 << input_depth) - 1);
const double by = kb * 219 * (1 << (output_depth - 8)) / ((1 << input_depth) - 1);

const double ys = 16 * (1 << (output_depth - 8)) + .5;

const double rcb = -kr / ((1 - kb) * 2) * 244 * (1 << (output_depth - 8)) / (((1 << input_depth) - 1) * 4);
const double gcb = -(1 - kr - kb) / ((1 - kb) * 2) * 244 * (1 << (output_depth - 8)) / (((1 << input_depth) - 1) * 4);
const double bcb = (1 - kb) / ((1 - kb) * 2) * 244 * (1 << (output_depth - 8)) / (((1 << input_depth) - 1) * 4);

const double rcr = (1 - kr) / ((1 - kr) * 2) * 244 * (1 << (output_depth - 8)) / (((1 << input_depth) - 1) * 4);
const double gcr = -(1 - kr - kb) / ((1 - kr) * 2) * 244 * (1 << (output_depth - 8)) / (((1 << input_depth) - 1) * 4);
const double bcr = -kb / ((1 - kr) * 2) * 244 * (1 << (output_depth - 8)) / (((1 << input_depth) - 1) * 4);

const double cs = 128 * (1 << (output_depth - 8)) + .5;

int x, y, frame;

for (frame = 0; 1; frame++) {
for (y = 0; y < height; y += 2) {

// reads two rows at once
if (fread(rgb_buf, sizeof(rgb_buf[0]), width * 2 * 3, stdin) != width * 2 * 3) {
print_end_msg(frame);
return 0;
}

for (x = 0; x < width * 2; x++)
y_buf[x] = rgb_buf[x * 3 + 0] * ry + rgb_buf[x * 3 + 1] * gy + rgb_buf[x * 3 + 2] * by + ys;

if (fwrite(y_buf, sizeof(y_buf[0]), width * 2, stdout) != width * 2) {
print_end_msg(frame);
return 0;
}

for (x = 0; x < width; x += 2) {
int r, g, b;
r = rgb_buf[x * 3 + 0] + rgb_buf[x * 3 + 3] + rgb_buf[(x + width) * 3 + 0] + rgb_buf[(x + width) * 3 + 3];
g = rgb_buf[x * 3 + 1] + rgb_buf[x * 3 + 4] + rgb_buf[(x + width) * 3 + 1] + rgb_buf[(x + width) * 3 + 4];
b = rgb_buf[x * 3 + 2] + rgb_buf[x * 3 + 5] + rgb_buf[(x + width) * 3 + 2] + rgb_buf[(x + width) * 3 + 5];

cb_buf[x / 2 + (y / 2 * width / 2)] = r * rcb + g * gcb + b * bcb + cs;
cr_buf[x / 2 + (y / 2 * width / 2)] = r * rcr + g * gcr + b * bcr + cs;
}
}

if (fwrite(cb_buf, sizeof(cb_buf[0]), (width / 2) * (height / 2), stdout) != (width / 2) * (height / 2)) {
print_end_msg(frame);
return 0;
}

if (fwrite(cr_buf, sizeof(cr_buf[0]), (width / 2) * (height / 2), stdout) != (width / 2) * (height / 2)) {
print_end_msg(frame);
return 0;
}
}

return 0;
}


rgb48le_to_yuv420p16le 1280 720 < rgb48.raw | x264 - -o video3.mkv --demuxer raw --input-depth 16 --input-res 1280x720 --fps 25 --preset slow -q 0

raw [info]: 1280x720p 0:0 @ 25/1 fps (cfr)
x264 [info]: using cpu capabilities: MMX2 SSE2Slow SlowCTZ
x264 [info]: profile High 4:4:4 Predictive, level 3.1, bit depth 8
9 frames: 2.60 fps, 18593.07 kb/s
rgb48le_to_yuv420p16le converted 10 frames
x264 [info]: frame I:1 Avg QP: 0.00 size:135079
x264 [info]: frame P:9 Avg QP: 0.00 size: 87508
x264 [info]: mb I I16..4: 80.1% 18.7% 1.2%
x264 [info]: mb P I16..4: 3.3% 0.5% 0.0% P16..4: 66.5% 18.2% 10.5% 0.0% 0
.0% skip: 1.0%
x264 [info]: 8x8 transform intra:17.5% inter:56.4%
x264 [info]: coded y,uvDC,uvAC intra: 92.3% 87.6% 87.5% inter: 57.9% 89.9% 89.8%

x264 [info]: i16 v,h,dc,p: 28% 0% 60% 12%
x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 25% 1% 48% 6% 8% 5% 2% 2% 3%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 39% 12% 41% 2% 3% 1% 1% 0% 1%
x264 [info]: i8c dc,h,v,p: 52% 1% 34% 13%
x264 [info]: Weighted P-Frames: Y:0.0% UV:0.0%
x264 [info]: ref P L0: 67.5% 0.0% 19.7% 7.7% 3.3% 1.8%
x264 [info]: kb/s:18452.96

encoded 10 frames, 2.55 fps, 18463.72 kb/s

Now the video looks better: http://img219.imageshack.us/img219/4241/video3e.png

[CruNcher]

is the rgb48 colorspace conversion even part of x264 or isn't it ffmpegs ?

[LoRd_MuldeR]

is the rgb48 colorspace conversion even part of x264 or isn't it ffmpegs ?

libswscale, I think.

[J_Darnley]

It is all libswscale

[fransky]

althought x264 doesn't allow to tweak aq-strength in --zones, but it doesn't give any warning or error like "you can't change aq in --zones" when such settings is given.

[Emulgator]

From
http://forum.doom9.org/showthread.php?t=162200
(My post #12 in this thread):
______________________________________________________________________________________

So this is Scubasteve2365's error message of Scenarist BD 5.1.3 when remuxing (I assume)
compliantly encoded elementary video that has been demuxed and cut using tsMuxeR:

ERROR|MUX_SN_E_TS_UNKNOW_ERR|P:\BD Demo Vol2\Scenarist Project\ProjectName\Test Project\02.00.0000\Output\MUX\BDRE\DB\BDMV\STREAM/00031.m2ts|0|Unknown Error|TSWrapper.dll::CTSWrapper::ProcThreadMain::This program has a bug. - m_ptsOfNextGOP is empty.|

The following is the error message of my DVD-Architect Pro 5.0.and 5.2
when attempting to mux x264-encoded interlaced elementary video:

File name: STREAM/00000.m2ts
Status: TSWrapper.dll::CTSWrapper::ProcThreadMain::This program has a bug. - m_ptsOfNextGOP is empty.

Sounds familiar. The same wording, letter by letter.
Scenarist and Sony DVD-A, they both seem to use the same muxer.

(My TSWrapper.dll in both DVD-A 5.0 and 5.2 is version 2.0.0.1 (2.0.0.0001),
Product name: mux, XML 1.0.0.0001, Copyright 2005, Size 561.152 Bytes)
____________________________________________________________________________________

Can it still be that x264-encoded streams that contain interlaced flags
might miss a certain condition of embedded parameters only at their end(s) ?
(The muxing failure is exhibited on fake-interlaced x264 encodes as well.)

The OP tells that:


The video is from a commerical blu-ray release.
I take small 3 minute or so segments from blu-rays for the purposes of demoing home theater equipment.
I simply ripped the movie to harddisk,
used TSmuxer to trim the clip to the desired start/end point and then demuxed with eac3to.
Perhaps TSMuxer damaged the trimmed clip. I've read that I shouldn't use TSMuxer for this purpose
but there isn't a lot of software out there, that I've found, that will nicely trim M2TS files.


This might suggest that trimming of an already muxing-compliant .m2ts
may lead to inconsistencies of embedded stream parameters close to the stream's end(s)
and in consequence refusal by the muxer engine of Scenarist.

SONY DVD-A Pro 5.0 and 5.2. throw the same muxer fault (word by word) when muxing x264 interlaced.
Maybe the same reason? (Inconsistencies of embedded stream parameters close to the stream's end(s))

BTW, until today I got no response from sonycreativesoftware.com about this.

The muxer fault is still there with interlaced and x264 r2074, just checked today.

[mp3dom]

Have you tried to trim the clip using Scenarist CC6 or trimming at clip level rather than using TSMuxer? By the way, both Scenarist and DVDArchitect uses the Sony muxer engine.

[Emulgator]

Thanks, mp3dom. Good to hear that my guessing about the muxer was correct though.
I have no access to Scenarist BD at the moment, but Sony DVD-A Pro 5.0+5.2, Adobe Encore CS3, DVDitProHD 6.4.

I just relied on the OP's (Scubasteve2365) findings regarding Scenarist BD.
I did not perform any trimming tests at all, I was just drawing conclusions from his results.
(I edited my post now for clarification)

So it may well be the case that interlaced x264 streams will not pass Scenarist BD as well.
The successful x264 authored BDs all seem to have had progressive content then ?

[sneaker_ger]

Quote from x264bluray.com (http://www.x264bluray.com/issues-with-certain-players):
Sony DVD-A does not support interlaced files encoded with x264.

No idea if it still applies or if it's related to your issue. Maybe some dev can comment.

[Emulgator]

I actually reported this case myself to kieranrk back then.

In between I got somebody to peek at some dlls.
I have been told that TSWrapper.dll in Scenarist BD 5.2 has the same Product Name "mux",
a size of 557.056 Bytes, shows File Version 2.4.0.6, XML is Version 2.0.0.0001, Copyright 2005-2009,
so has been developed further than the version in SONY DVD-A.

And TSWrapper.dll in my almost unusable DVDitProHD 6.4 has
Product Name "mux", size 552.960 Bytes, shows File Version 1.0.1.0 and Copyright 2005.

This is the muxer story so far.

[mp3dom]

Yes, the Scenarist muxer engine comes from Sony but it's a more advanced/up to date version than what you can find in DVD-Architect. I think Scenarist shares the exact same muxer as (Sony) Blu-Print

[Chumbo]

I was doing some encoding today and noticed that x264_64.exe keeps rebuilding the index file. I'm using CLI and my input file is an MKV. If the related .ffindex file exists would it not use it? I also tried specifically using the --index and that doesn't work either. I checked the long help and could not find any specific switch that would use the existing .ffindex file so it doesn't have to reindex every time I start the command. Here's an example of my command line:"x264_64.exe" --index "mediafile.mkv.ffindex" --pass 1 --bitrate 3000 --fps 24000/1001 --vf resize:width=1280,height=720,method=bicubic --stats "mediafile.stats" --open-gop --output NUL "mediafile.mkv"Looks like I'm using this version: x264 core:116 r2044 392e762

[EDIT] Never mind on this now. Not sure why, but now it's working fine. I don't even know what I did, but my cmd file is now running the x264 command and it's not rebuilding the index.

[EDIT2] No sorry, it's not working actually, so I still need help getting the x264_64.exe to recognize the existing .ffindex file so it doesn't have to index the input file. Any help is greatly appreciated.

[MasterNobody]

IIRC you need that ffms version used for indexing (as I understand you used standalone indexer to create index-file, not x264) and compiled in x264 was exactly same (and so have same format of index-file). Also there is check that index file must be newer than file opened.

[Chumbo]

IIRC you need that ffms version used for indexing (as I understand you used standalone indexer to create index-file, not x264) and compiled in x264 was exactly same (and so have same format of index-file). Also there is check that index file must be newer than file opened.
Yes, I have a batch file that runs several processes before doing the encoding, one of which is the indexing, e.g., ffmsindex -f -m lavf "mediafile.mkv"
I used Process Monitor to see which ffms DLL was being used by the x264 executable but none show up. So I guess the ffms library is compiled into the executable? So if my standalone version of the indexer doesn't match the library used by x264_64.exe then there's no use indexing with it?

[EDIT] Okay, I figured out the best way to approach this. When I use avs files as input, I'll just index the file myself. If I use MKV files as input, I'll just let x264 create/use the index file. That works well.

[Juce]

x264 --output-csp rgb bus_cif.y4m -o bus_rgb.mkv
ffplay bus_rgb.mkv

http://img228.imageshack.us/img228/8862/busrgb.png

Is this bug in x264 or ffplay?

x264 r2106 http://x264.nl
FFmpeg git-8475ec1 32-bit Static (Latest) (2011-10-31) http://ffmpeg.zeranoe.com/builds/

[Juce]

Looks like that ffmpeg does not detect the color space: x264 bus_rgb.mkv -o video.mkv

ffms [info]: 352x288p 35:32 @ 30/1 fps (vfr)
resize [warning]: converting from yuvj444p to yuv444p
resize [warning]: converting from yuv444p to yuv420p
x264 [info]: using SAR=35/32
x264 [info]: using cpu capabilities: MMX2 SSE2 SSE3 Cache64
x264 [info]: profile High, level 1.3
x264 [info]: frame I:1 Avg QP:28.20 size: 24834
x264 [info]: frame P:75 Avg QP:27.99 size: 8971
x264 [info]: frame B:74 Avg QP:32.54 size: 2291
x264 [info]: consecutive B-frames: 1.3% 98.7% 0.0% 0.0%
x264 [info]: mb I I16..4: 1.3% 47.0% 51.8%
x264 [info]: mb P I16..4: 0.3% 1.0% 1.4% P16..4: 32.1% 31.0% 30.7% 0.0% 0
.0% skip: 3.5%
x264 [info]: mb B I16..4: 0.0% 0.0% 0.0% B16..8: 38.5% 11.5% 5.5% direct:
9.5% skip:34.9% L0:31.2% L1:38.1% BI:30.7%
x264 [info]: 8x8 transform intra:40.5% inter:56.8%
x264 [info]: coded y,uvDC,uvAC intra: 86.5% 98.9% 94.6% inter: 28.3% 60.6% 53.5%

x264 [info]: i16 v,h,dc,p: 14% 70% 8% 8%
x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 18% 19% 30% 6% 4% 4% 6% 6% 7%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 28% 28% 19% 4% 5% 4% 4% 4% 5%
x264 [info]: i8c dc,h,v,p: 39% 28% 26% 8%
x264 [info]: Weighted P-Frames: Y:4.0% UV:4.0%
x264 [info]: ref P L0: 65.7% 13.8% 14.0% 6.2% 0.3%
x264 [info]: ref B L0: 93.4% 6.6%
x264 [info]: kb/s:1387.46

encoded 150 frames, 21.57 fps, 1388.58 kb/s

[Dark Shikari]

ffmpeg doesn't support planar RGB yet.

[Snowknight26]

It does now?

http://git.videolan.org/?p=ffmpeg.git;a=commit;h=1e79926f9e8bbfa2920a1d40e36d9ffcf244fe0a

[Chikuzen]

ffmpeg support planar RGB, but libav doesn't yet.
x264.exe provided by x264.nl is using libav's libraries.

edit:
sorry, i mistook.
that patch is for planar RGB to packed RGB conversion.
ffmpeg still doesn't support packed RGB to planar RGB.

[Chumbo]

I recently encoded a batch of files and then noticed some were missing. After further investigation, I noticed that those files had UTF8 characters in the file names which x264 would not recognize and blow chunks. Any chance of updating the 32/64bit executables to support UTF8 characters please? If a title has the ?, for example, I can't use this due to it being a reserved OS character, so I use the upside down ? (¿). I also tend to use the upside down exclamation point (¡) just to avoid the pitfalls of command-line processing with it.

[LoRd_MuldeR]

As far as I see, x264 uses the plain "char" type all the way. This will support Unicode just fine on Linux, as Linux uses UTF-8 all the way. Also in its API functions!

On Windows however, the local ANSI Codepage will be used for char-strings. This means: Everything that doesn't fit into the local Codepage is already "messed up" when it arrives at the "main" function.

Also, even if you use UTF-8 inside your application, functions like fopen() won't be able to deal with UTF-8 encoded strings - on Windows.


There are basically two workarounds:

(1) Use the wchar_t type with UTF-16 all the way, on the Windows platform. This means you will have to use wmain() instead of main(), wfopen() instead of fopen(), wcslen() instead of strlen() and so on.

(2) Write short wrapper functions that convert from/to UTF-16 to/from UFT-8 for the Windows platform. This way you can keep everything to the char type with UFT-8 inside your own application.


Solution (1) requires the use of platform-specific macros all over the program code. At least if you want to support Windows (wchar_t/UTF-16) as well as Linux (char/UTF-8) with the same unmodified code.

At the same time with solution (2) 99% of your code can remain unchanged, but for every API call that goes outside your own code (and uses strings), you'll have to convert back to UTF-16 - on Windows.


I have successfully applied method (2) to a bunch of Linux tools that had been ported to Windows, but without Unicode support. It usually is not a big deal and I guess the same could be done with x264...


That's my "Unicode Support" stuff, I usually hack into Linux-style C programs:
* http://pastie.org/private/hnevzpd1cu5zv04ovptjbq
* http://pastie.org/private/acu7qmfrcyvdswmwmcjaw

And this is how you would have to change the main method to make this stuff work:
* http://pastie.org/private/uwavpv4vosjckawh0whpwa


If there is any change to get Win32 Unicode support committed into x264 (and if really nobody else has done this yet), I would be happy to submit a patch.

[kemuri-_9]

What you're only really caring about is that filenames can be utf-8, not the entirety of x264.

Updating the entirety of x264 to be utf-8 on windows is largely more extensive that what LoRd_MuldeR's patches indicate....
his patch even opens up a can of worms for placing utf-8 strings into functions that heavily expect ansi strings, which is going to cause fun issues later.

I am somehow recalling a patch from one of the x264 licensees on the subject, but not sure what happened to that exactly. Guessing it was forgotten due to lack of interest.

[LoRd_MuldeR]

Well, the good thing about UTF-8 is that all ASCII characters (which includes the "control characters", such as '\n' and friends) are identical between ANSI and UTF-8.

Moreover, all UTF-8 characters, that can not be represented in plain ASCII, are coded in a way that they cannot be confused with ASCII characters. Thus all the C string functions deal properly with UTF-8.

Also keep in mind that x264 actually is using UTF-8 encoded strings on the Linux platform, since the beginning of time. AFAIK what you gent as agrv[] in your main() method is UTF-8, on Linux.

Printing out Unicode characters properly to the Windows console, may they be encoded as UTF-8 or UTF-16 or whatever, is another funny story and probably not the most important thing to worry about ;)

BTW: The code snippets above are not patches for x264, they just give the idea how a simple patch could be made. I patched like probably a dozen tools that way and it seems to work pretty well...

(Actually this is the way how LAME does handle Unicode on the Win32 platform. Looking at the LAME code gave me the initial idea on how to do it)

[kemuri-_9]

Yes, the basic ASCII characters 0 - 0x7f are generally the same across all codepages.

However, the 0x80 to 0xff characters are primarily different for each local codepage, additionally differing from what UTF-8 has for these.

This is the 'can of worms' I was referring to.

[Dark Shikari]

The patch from Pegasys was for UTF-16 support.

[LoRd_MuldeR]

If anybody cares, I have hacked together a quick UTF-8 patch for Win32.

http://img46.imageshack.us/img46/4205/cwindowssystem32cmdexe2.th.png (http://img46.imageshack.us/img46/4205/cwindowssystem32cmdexe2.png)

[Chumbo]

What you're only really caring about is that filenames can be utf-8, not the entirety of x264.
...
That's the case for me so it can process my files named with UTF8 characters.

If anybody cares, I have hacked together a quick UTF-8 patch for Win32.

http://img46.imageshack.us/img46/4205/cwindowssystem32cmdexe2.th.png (http://img46.imageshack.us/img46/4205/cwindowssystem32cmdexe2.png)
Sweet! :) Any chance of getting this into the latest dev build? I'm on 64bit so can the 64bit be patched likewise please?

[LoRd_MuldeR]

Also added a workaround for Avisynth (and FFMS2), which, as far as I know, has no way of properly passing Unicode strings.

(Updated patch in previous post)

[LoRd_MuldeR]

Added a workaround for MP4 output too. The GPAC library doesn't support Unicode either.

(Updated patch in previous post)

[LoRd_MuldeR]

Okay, FFMS2 does support Unicode/UTF-8 paths, if initialized accordingly. Unfortunately its UTF-8 support is broken for the index file, so we still need the workaround for the index file.

(Updated patch in previous post)

[kemuri-_9]

All this is only proving is that dealing with anything non-ascii is just not worth it on windows.

it's easy enough to just rename your files...

[LoRd_MuldeR]

Well, the lack of proper Unicode support may be a negligible problem for US users, but it certainly is a big annoyance for people from certain other countries :o

Not supporting Unicode is behind the times (this is not the early Nineties anymore). And adding proper UFT-8 support for Win32 is possible - with very minor code change*.

I see few reasons to not do it. The biggest problem, at the moment, is that some third-party libs, such as Avisynth and FFMS2, screw up :scared:

While I don't think there is much hope for Avisynth, FFMS2 seems to be under active development and already does support UTF-8 - just not for the index file (yet)**.

And even for those third-party libs that we can't get to work with UTF-8, we can still convert to "short" path names internally, which works most of the time.

(I know that the generation of "short" path names can be disabled by registry hack, but I'm yet to encounter such a system. And in that very rare case, we can still rename)

__________________
(*) Most code changes in the patch are actually workarounds for third-party libs, that we hopefully can get rid of (not the libs, the workarounds), as soon as those libs are fixed
(**) It's actually a MinGW-specific problem with the fstream class. The workaround is creating your own implementation of fstream that uses wfopen() internally. I have done that for a project at work already.

[Juce]

A video is bluish if a source is 48 bit per pixel PNG.

x264 r2145 (x264.nl), Windows XP 32 bit

x264 "frame%03d.png" -o testi.mkv 2> log.txt

ffms [error]: could not create index
lavf [info]: 1280x720p 0:1 @ 25/1 fps (vfr)
resize [warning]: converting from rgb48be to rgb48le
resize [warning]: converting from rgb48le to yuv420p16le
x264 [info]: using cpu capabilities: MMX2 SSE2Slow SlowCTZ
x264 [info]: profile High, level 3.1
1 frames: 0.82 fps, 1240.40 kb/s
4 frames: 2.64 fps, 567.35 kb/s
7 frames: 3.73 fps, 464.89 kb/s
10 frames: 4.51 fps, 403.86 kb/s

x264 [info]: frame I:1 Avg QP:18.33 size: 5501
x264 [info]: frame P:6 Avg QP:19.39 size: 1946
x264 [info]: frame B:3 Avg QP:21.61 size: 771
x264 [info]: consecutive B-frames: 40.0% 60.0% 0.0% 0.0%
x264 [info]: mb I I16..4: 72.6% 26.6% 0.8%
x264 [info]: mb P I16..4: 10.2% 1.2% 0.0% P16..4: 15.2% 0.7% 2.0% 0.0% 0.0% skip:70.7%
x264 [info]: mb B I16..4: 2.9% 0.4% 0.0% B16..8: 12.9% 0.4% 0.0% direct: 0.1% skip:83.3% L0:30.9% L1:68.9% BI: 0.3%
x264 [info]: 8x8 transform intra:19.8% inter:99.7%
x264 [info]: coded y,uvDC,uvAC intra: 8.0% 25.9% 1.5% inter: 2.4% 9.2% 0.0%
x264 [info]: i16 v,h,dc,p: 43% 49% 2% 6%
x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 2% 12% 84% 0% 0% 0% 0% 0% 0%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 13% 14% 72% 0% 0% 0% 0% 0% 0%
x264 [info]: i8c dc,h,v,p: 49% 43% 7% 1%
x264 [info]: Weighted P-Frames: Y:0.0% UV:0.0%
x264 [info]: ref P L0: 84.4% 0.6% 12.5% 2.6%
x264 [info]: ref B L0: 45.3% 54.7%
x264 [info]: kb/s:389.84

encoded 10 frames, 4.51 fps, 403.86 kb/shttp://img27.imageshack.us/img27/641/png48src.th.png (http://imageshack.us/photo/my-images/27/png48src.png/)

PNG-images: http://www.mediafire.com/file/vc5nkbnakw627y9/frames.zip

[MasterNobody]

A video is bluish if a source is 48 bit per pixel PNG.

x264 r2145 (x264.nl), Windows XP 32 bit

x264 "frame%03d.png" -o testi.mkv 2> log.txt
...
http://img27.imageshack.us/img27/641/png48src.th.png (http://imageshack.us/photo/my-images/27/png48src.png/)

PNG-images: http://www.mediafire.com/file/vc5nkbnakw627y9/frames.zip
That is bug in lavf input (ffmpeg/libav) because there are same artefacts with:
avconv -i "frame%03d.png" -vcodec huffyuv test.avi

[professor_desty_nova]

I just noticed a difference when when input is 10bit avc with the 8bit version of x264:

with x264 revision 2120 the resize warning is: "resize [warning]: converting from yuv420p10le to yuv420p"

with x264 revision 2146 the resize warning is: "resize [warning]: converting from yuv420p10le to yuv420p16le"

Is this a bug? From the warning it seems now it's converting from 10 bit to 16bit prior to encoding in the 8bit version of x264!

I don't know if it's related to this:
If the input is 8bit, both 2120 and 2146 use the same number of I,P and B-frames.
If the input is 10bit, the resulting encode of 2120 and 2146 has a different number of I, P and B-frames.

[kemuri-_9]

This is actually not a bug, the old functionality was instead a bug:

Upon receiving input from ffmpeg/libav (whether via libavformat or ffms), the input needs to be 'normalized' into something the x264cli system can recognize.

for 8bit and 16bit per channel formats, x264cli supports these naturally (for what it does support), so no conversion is actually required.
However any intermediate bit/channel formats such as 10bit/channel need to be normalized.

Previously, most, if not all, types of such formats were normalized to 8bit. This is actually a quality degradation, so it was corrected to normalize the formats to 16bit, to allow for retention of quality when passing the data into the filtering system.

using an input -> intermediate -> final (libx264) denotation,
10 -> 8 -> 10 - what is passed to libx264 is not the same quality that was originally read in, but
10 -> 16 -> 10 has the same quality, as there was no bit-depth drop below what the input has.

[professor_desty_nova]

This is actually not a bug, the old functionality was instead a bug:

Upon receiving input from ffmpeg/libav (whether via libavformat or ffms), the input needs to be 'normalized' into something the x264cli system can recognize.

for 8bit and 16bit per channel formats, x264cli supports these naturally (for what it does support), so no conversion is actually required.
However any intermediate bit/channel formats such as 10bit/channel need to be normalized.

Previously, most, if not all, types of such formats were normalized to 8bit. This is actually a quality degradation, so it was corrected to normalize the formats to 16bit, to allow for retention of quality when passing the data into the filtering system.

using an input -> intermediate -> final (libx264) denotation,
10 -> 8 -> 10 - what is passed to libx264 is not the same quality that was originally read in, but
10 -> 16 -> 10 has the same quality, as there was no bit-depth drop below what the input has.

So, if you are using the 8bit x264 binary and start with 10bit source, it now does something like 10 -> 16 ->8.

[LoRd_MuldeR]

So, if you are using the 8bit x264 binary and start with 10bit source, it now does something like 10 -> 16 ->8.

Yes, I think so.

[kemuri-_9]

So, if you are using the 8bit x264 binary and start with 10bit source, it now does something like 10 -> 16 ->8.

Yes, this is correct.

it might be expected that a
resize [warning]: converting from yuv420p16le to yuv420p
line would exist to indicate this, but the down conversion is actually handled separately from libswscale, so there is no message.

[professor_desty_nova]

So, I guess the difference I'm seeing (If the input is 10bit, the resulting encode of 2120 and 2146 has a different number of I, P and B-frames and with the same crf the 2146 file is bigger, with 8bit input it stays the same between versions) is because now the down-conversion is handled by a different part of the x264 binary, and the result that is fed to the encoder part is not the same.

[kemuri-_9]

So, I guess the difference I'm seeing (If the input is 10bit, the resulting encode of 2120 and 2146 has a different number of I, P and B-frames and with the same crf the 2146 file is bigger, with 8bit input it stays the same between versions) is because now the down-conversion is handled by a different part of the x264 binary, and the result that is fed to the encoder part is not the same.

Yes, previously you were having libswscale perform a 10->8 conversion and that's it.
Now you're having libswscale perform a 10->16 conversion and x264cli perform a 16->8 dither.

It should be noted that there is currently a development patch to fix the dithering, so after that goes in you'll likely see the output statistics change again.

[professor_desty_nova]

OK, I guess I'll wait for the fix to the dithering, since when I compared between versions 2120 and 2146, to me 2120 seemed better.