Out of memory (x264)

[cdz]

Just want to clear up on why this is happening.
It's on an AMD-based 64bit machine with 4GB memory.
Either the kernel kills it, or it just stalls with "Unable to fork: Cannot allocate memory".

While I understand that using the slow presets (rc-lookahead etc.), it seems a bit weird for the program to eat all the memory without any control mechanism preventing it.

The source material is 1920x1200, with presets slow/slower.
Since I'm batch converting a bunch of various stuff with the same settings, it's a bit annoying when it stops halfway through the night.

Getting more RAM would obviously fix this, but why exactly does this happen, and what are the upper boundaries for memory consumption on x264?
Any way to estimate this given resolution/settings/motion rates?

Thanks!!:)

[Dark Shikari]

x264 allocates all its memory at the start of encoding, so if you're having trouble later on, the problem is likely somewhere else (e.g. in a decoder, Avisynth filter, etc).

[cdz]

Hi - thanks for replying shiki :)

Yes, this is indeed strange. I've never had any troubles during years I've encoded with x264.

I'm now compiling everything from scratch without x264 dependency, as the latest (arch) packages either wants libx264.so.129 or 130.. And all I want ffmpeg anyway is lavf support.

I can only assume the problem is because I symlinked so.130 to so.129 to clear the errors. Let's see how it goes!

[cdz]

Still none the wiser after a few more tests

Fresh up-to-date arch install
Tried several recent x264 snapshots.
Compiled against ffmpeg 1.2.1, and most recent git.

Tested through ffmpeg, and works - but not directly with x264.

Also, this seems to happen with all clips that exceed ~10GB or > 1.5k frames


ffmpeg

./configure \
--prefix=/usr \
--disable-debug \
--enable-libmp3lame \
--enable-libxvid \
--enable-gpl


x264

.configure --bit-depth=10 --enable-static --enable-shared


x264 log

[avi @ 0x26f84c0] non-interleaved AVI
lavf [info]: 1920x1200p 0:1 @ 0/0 fps (vfr)
resize [warning]: converting from bgr24 to yuv420p
x264 [debug]: using mv_range_thread = 200
x264 [info]: using cpu capabilities: MMX2 SSE2Slow SlowCTZ
x264 [info]: profile High 10, level 5.0, 4:2:0 10-bit
x264 [debug]: frame= 0 QP=32.00 NAL=3 Slice:I Poc:0 I:9000 P:0 SKIP:0 size=549391 bytes
x264 [debug]: frame= 1 QP=40.00 NAL=2 Slice:P Poc:2 I:64 P:644 SKIP:8292 size=27678 bytes
x264 [debug]: frame= 2 QP=38.00 NAL=2 Slice:P Poc:4 I:101 P:815 SKIP:8084 size=37884 bytes
x264 [debug]: frame= 3 QP=36.00 NAL=2 Slice:P Poc:6 I:94 P:985 SKIP:7921 size=48350 bytes
x264 [debug]: frame= 4 QP=35.00 NAL=2 Slice:P Poc:8 I:76 P:1078 SKIP:7846 size=52357 bytes
x264 [debug]: frame= 5 QP=35.00 NAL=2 Slice:P Poc:10 I:69 P:1032 SKIP:7899 size=50569 bytes
x264 [debug]: frame= 6 QP=34.00 NAL=2 Slice:P Poc:12 I:78 P:1197 SKIP:7725 size=58025 bytes
x264 [debug]: frame= 7 QP=34.00 NAL=2 Slice:P Poc:14 I:68 P:1053 SKIP:7879 size=54543 bytes
x264 [debug]: frame= 8 QP=34.00 NAL=2 Slice:P Poc:16 I:64 P:1091 SKIP:7845 size=56695 bytes
x264 [debug]: frame= 9 QP=34.00 NAL=2 Slice:P Poc:18 I:93 P:1107 SKIP:7800 size=60375 bytes
x264 [debug]: frame= 10 QP=34.00 NAL=2 Slice:P Poc:20 I:65 P:1068 SKIP:7867 size=57413 bytes
[.......]
x264 [debug]: frame=1943 QP=37.00 NAL=2 Slice:P Poc:386 I:285 P:2723 SKIP:5992 size=98249 bytes
x264 [debug]: frame=1944 QP=37.00 NAL=2 Slice:P Poc:388 I:259 P:2958 SKIP:5783 size=101068 bytes
x264 [debug]: frame=1945 QP=37.00 NAL=2 Slice:P Poc:390 I:293 P:2973 SKIP:5734 size=103231 bytes

Killed


dmesg

[10336.190740] x264 invoked oom-killer: gfp_mask=0x280da, order=0, oom_score_adj=0
[10336.190747] x264 cpuset=/ mems_allowed=0
[10336.190750] Pid: 5494, comm: x264 Tainted: G O 3.9.7-1-ARCH #1
[10336.190752] Call Trace:
[10336.190763] [<ffffffff814caf13>] dump_header+0x7f/0x200
[10336.190778] [<ffffffff811250d6>] oom_kill_process+0x206/0x3a0
[10336.190783] [<ffffffff81125857>] out_of_memory+0x427/0x470
[10336.190796] [<ffffffff8112b2a4>] __alloc_pages_nodemask+0x9c4/0xa70
[10336.190802] [<ffffffff8116950a>] alloc_pages_vma+0x9a/0x190
[10336.190806] [<ffffffff8114be87>] handle_pte_fault+0x777/0xaa0
[10336.190830] [<ffffffff8114d92d>] handle_mm_fault+0x2bd/0x650
[10336.190843] [<ffffffff814d5fb0>] __do_page_fault+0x1d0/0x5c0
[10336.190869] [<ffffffffa035d6fa>] ? VBoxGuestCommonIOCtl+0x27a/0x1700 [vboxguest]
[10336.190880] [<ffffffff814d63ae>] do_page_fault+0xe/0x10
[10336.190884] [<ffffffff814d3088>] page_fault+0x28/0x30
[10336.190890] [<ffffffff8127a070>] ? copy_user_generic_string+0x30/0x40
[10336.190895] [<ffffffffa0022fb3>] ? sf_reg_read+0xe3/0x170 [vboxsf]
[10336.190901] [<ffffffff8118b738>] vfs_read+0x98/0x170
[10336.190904] [<ffffffff8118bb09>] sys_read+0x49/0xa0
[10336.190910] [<ffffffff814da19d>] system_call_fastpath+0x1a/0x1f
[10336.190912] Mem-Info:
[10336.190915] Node 0 DMA per-cpu:
[10336.190918] CPU 0: hi: 0, btch: 1 usd: 0
[10336.190920] CPU 1: hi: 0, btch: 1 usd: 0
[10336.190922] CPU 2: hi: 0, btch: 1 usd: 0
[10336.190924] CPU 3: hi: 0, btch: 1 usd: 0
[10336.190926] Node 0 DMA32 per-cpu:
[10336.190928] CPU 0: hi: 186, btch: 31 usd: 29
[10336.190931] CPU 1: hi: 186, btch: 31 usd: 27
[10336.190933] CPU 2: hi: 186, btch: 31 usd: 30
[10336.190935] CPU 3: hi: 186, btch: 31 usd: 5
[10336.190939] active_anon:362585 inactive_anon:48 isolated_anon:0
active_file:59 inactive_file:0 isolated_file:0
unevictable:0 dirty:0 writeback:0 unstable:0
free:12743 slab_reclaimable:2573 slab_unreclaimable:2344
mapped:113 shmem:86 pagetables:1627 bounce:0
free_cma:0
[10336.190942] Node 0 DMA free:6408kB min:464kB low:580kB high:696kB active_anon:9440kB inactive_anon:0kB \
active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB \
present:15992kB managed:15908kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB \
slab_reclaimable:0kB slab_unreclaimable:8kB kernel_stack:0kB pagetables:20kB unstable:0kB \
bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 \all_unreclaimable? yes
[10336.190948] lowmem_reserve[]: 0 1491 1491 1491
[10336.190950] Node 0 DMA32 free:44564kB min:44588kB low:55732kB high:66880kB active_anon:1440900kB \
inactive_anon:192kB active_file:236kB inactive_file:0kB unevictable:0kB isolated(anon):0kB \
isolated(file):0kB present:1568704kB managed:1527056kB mlocked:0kB dirty:0kB writeback:0kB \
mapped:452kB shmem:344kB slab_reclaimable:10292kB slab_unreclaimable:9368kB kernel_stack:752kB \
pagetables:6488kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:1076 all_unreclaimable? yes
[10336.190955] lowmem_reserve[]: 0 0 0 0
[10336.190958] Node 0 DMA: 0*4kB 1*8kB (U) 0*16kB 0*32kB 2*64kB (U) 1*128kB (U)\
2*256kB (UM) 1*512kB (M) 1*1024kB (U) 2*2048kB (MR) 0*4096kB = 6408kB
[10336.190964] Node 0 DMA32: 351*4kB (UEM) 203*8kB (UEM) 220*16kB (UEM) 116*32kB (UEM) 56*64kB (UEM) 32*128kB (UEM) \
22*256kB (UE) 13*512kB (UE) 14*1024kB (UEM) 0*2048kB 0*4096kB = 44564kB
[10336.190972] 214 total pagecache pages
[10336.190981] 0 pages in swap cache
[10336.190984] Swap cache stats: add 0, delete 0, find 0/0
[10336.190986] Free swap = 0kB
[10336.190987] Total swap = 0kB
[10336.197045] 396271 pages RAM
[10336.197049] 9185 pages reserved
[10336.197066] 264845 pages shared
[10336.197069] 372773 pages non-shared
[10336.197071] [ pid ] uid tgid total_vm rss nr_ptes swapents oom_score_adj name
[10336.197102] [ 111] 0 111 7826 108 18 0 -1000 systemd-udevd
[10336.197112] [ 117] 0 117 22020 89 44 0 0 systemd-journal
[10336.197119] [ 192] 0 192 9935 138 23 0 -1000 sshd
[10336.197128] [ 193] 0 193 6526 69 18 0 0 systemd-logind
[10336.197132] [ 195] 81 195 4327 63 13 0 -900 dbus-daemon
[10336.197135] [ 198] 0 198 15239 108 36 0 0 login
[10336.197153] [ 209] 0 209 49789 373 95 0 0 nmbd
[10336.197157] [ 212] 0 212 69070 520 132 0 0 smbd
[10336.197176] [ 215] 1000 215 3932 150 12 0 0 bash
[10336.197180] [ 216] 0 216 69070 520 127 0 0 smbd
[10336.197198] [ 218] 0 218 22631 224 47 0 0 sshd
[10336.197202] [ 221] 1000 221 22631 223 46 0 0 sshd
[10336.197244] [ 222] 1000 222 3962 172 13 0 0 bash
[10336.197249] [ 712] 0 712 75829 633 145 0 0 smbd
[10336.197260] [ 2488] 0 2488 22631 222 49 0 0 sshd
[10336.197269] [ 2490] 1000 2490 22631 222 47 0 0 sshd
[10336.197273] [ 2491] 1000 2491 3961 168 12 0 0 bash
[10336.197299] [ 5494] 1000 5494 494696 359391 733 0 0 x264
[10336.197302] Out of memory: Kill process 5494 (x264) score 930 or sacrifice child
[10336.197476] Killed process 5494 (x264) total-vm:1978784kB, anon-rss:1437564kB, file-rss:0kB


kernel:

Linux 3.9.7-1-ARCH #1 SMP PREEMPT Thu Jun 20 22:45:32 CEST 2013 x86_64 GNU/Linux

/proc/cpuinfo
processor : 0
vendor_id : AuthenticAMD
cpu family : 16
model : 4
model name : AMD Phenom(tm) II X4 965 Processor
stepping : 3
cpu MHz : 3654.144
cache size : 512 KB
physical id : 0
siblings : 4
core id : 0
cpu cores : 4
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 5
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr \
pge mca cmov pat pse36 clflush mmx fsr sse sse2 ht \
syscall nx fxsr_opt rdtscp lm 3dnowext 3dnow \
ep_good nopl extd_apicid pni lahf_lm cp_legacy cr8_legacy
bogomips : 7311.80
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management:

[LoRd_MuldeR]

Tested through ffmpeg, and works - but not directly with x264.

So the problem does not occur if you use libx264 via FFmpeg, but it does occur with the x264 CLI encoder?

And if so, did you try using FFmpeg to decompress the input and then send it to x264 via pipe?

ffmpeg -i source.foo -f yuv4mpegpipe - | x264 -demuxer y4m <options> -

(BTW: What exactly is your source and your command-line?)

[cdz]

@mulder
I meant to edit the previous post, after ffmpeg startet malfunctioning when run with slower presets.

Piping seems to work:

(with -pix_ftmp yuv420p)
x264 [info]: frame I:92 Avg QP:31.76 size:599190
x264 [info]: frame P:22687 Avg QP:34.88 size: 90361
x264 [info]: mb I I16..4: 100.0% 0.0% 0.0%
x264 [info]: mb P I16..4: 5.0% 0.0% 0.0% P16..4: 39.1% 0.0% 0.0% 0.0% 0.0% skip:55.8%
x264 [info]: coded y,uvDC,uvAC intra: 62.5% 67.1% 51.1% inter: 23.9% 25.7% 13.9%
x264 [info]: i16 v,h,dc,p: 23% 39% 24% 13%
x264 [info]: i8c dc,h,v,p: 38% 34% 17% 11%
x264 [info]: kb/s:22179.91

encoded 22779 frames, 24.90 fps, 22179.92 kb/s


ffmpeg run with

-loglevel verbose -c:a flac -c:v libx264 -threads {3,auto} -preset {ultrafast,slow,slower}

Did a new round of tests just now, and hasn't crashed yet on either preset.

x264 Runs out of memory every time, when fed input directly.
(with --verbose --threads{3,auto} --preset{ultrafast,slow,slower})

Input is Fraps:

Video
Count : 263
Count of stream of this kind : 1
Kind of stream : Video
Kind of stream : Video
Stream identifier : 0
StreamOrder : 0
ID : 0
ID : 0
Format : Fraps
Commercial name : Fraps
Codec ID : FPS1
Codec ID/Url : http://www.fraps.com/
Codec : FPS1
Codec : FRAPS
Codec/Url : http://www.fraps.com/
Codec/CC : FPS1
Duration : 759300
Duration : 12mn 39s
Duration : 12mn 39s 300ms
Duration : 12mn 39s
Duration : 00:12:39.300
Bit rate : 643260937
Bit rate : 643 Mbps
Width : 1920
Width : 1 920 pixels
Height : 1200
Height : 1 200 pixels
Pixel aspect ratio : 1.000
Display aspect ratio : 1.600
Display aspect ratio : 16:10
Frame rate : 30.000
Frame rate : 30.000 fps
Frame count : 22779
Bits/(Pixel*Frame) : 9.306
Delay : 0
Delay : 00:00:00.000
Stream size : 61053503672
Stream size : 56.9 GiB (98%)
Stream size : 57 GiB
Stream size : 57 GiB
Stream size : 56.9 GiB
Stream size : 56.86 GiB
Stream size : 56.9 GiB (98%)
Proportion of this stream : 0.98303