Resource/memory allocation (x264)

[qyot27]

How exactly is this handled under a Linux system? The reason I ask is because while testing out my own builds of x264 on both my computer (Celeron II/PIII "Coppermine", 256 MB RAM, Ubuntu 8.10) and my grandparents' computer (Athlon64 "Orleans", 512 MB RAM, Ubuntu 8.10 64-bit), it seems to use way more resources than the Windows versions do, to the point that the mouse becomes just this side of unresponsive - something I have no memory of noticing when running it under Windows, even using the same encoder settings. It is true that the Linux builds report back higher fps values, which I assume is due to better resource management - if this aspect is just the way it is, then I can live with that.

But on the Orleans, going into standby (something I didn't expect to happen in the first place) seemed to lock up the resources in such a way that an encode that was going relatively smoothly - ~575 frames in about an hour and 15 minutes, as I was using insane settings - just about stopped, and spit out only about four more frames over the course of the next half hour; and this was after coming back out of standby. At least, I assume it was in standby - I'd shut off the screensaver, but the monitor had still shut off on its own and the power light was blinking as if it had gone on standby. It isn't like there was anything the matter with those frames either - I later encoded the same video under Windows and that frame range was encoded and passed just as fast as the rest of the video had been.


What sorts of things would cause this? Is it an issue with the compiler (I use gcc 4.3.2; the Windows builds are Skystrife's x86 builds, which use gcc 3.4.6), an underlying OS difference, or something else entirely? When compiling I do use -march=pentium3 and -march=athlon64 CFLAGS, respectively. Would using a different, light-weight environment (like Fluxbox, which I used to use as my main DE a couple years ago; these behaviors were observed under Gnome) be the solution? Anything other than just getting more RAM, basically.

[nm]

How exactly is this handled under a Linux system? The reason I ask is because while testing out my own builds of x264 on both my computer (Celeron II/PIII "Coppermine", 256 MB RAM, Ubuntu 8.10) and my grandparents' computer (Athlon64 "Orleans", 512 MB RAM, Ubuntu 8.10 64-bit), it seems to use way more resources than the Windows versions do, to the point that the mouse becomes just this side of unresponsive - something I have no memory of noticing when running it under Windows, even using the same encoder settings.
How much memory did x264 consume (ps aux or top, VSZ/VIRT and RSS/RES columns) and how much free memory was there available while running x264 (post the complete output of free)? Have you run x264 in Windows with the same hardware and same amount of RAM?

But on the Orleans, going into standby (something I didn't expect to happen in the first place) seemed to lock up the resources in such a way that an encode that was going relatively smoothly - ~575 frames in about an hour and 15 minutes, as I was using insane settings - just about stopped, and spit out only about four more frames over the course of the next half hour; and this was after coming back out of standby. At least, I assume it was in standby - I'd shut off the screensaver, but the monitor had still shut off on its own and the power light was blinking as if it had gone on standby.
Sounds like something got messed up when resuming. Maybe the video source got confused because of the time skip or the CPU frequency was reduced. Hard to say without investigating more while the machine is in that state. I doubt it was x264's fault.

Would using a different, light-weight environment (like Fluxbox, which I used to use as my main DE a couple years ago; these behaviors were observed under Gnome) be the solution? Anything other than just getting more RAM, basically.
Yes, if the system runs out of memory while running x264.

[qyot27]

How much memory did x264 consume (ps aux or top, VSZ/VIRT and RSS/RES columns) and how much free memory was there available while running x264 (post the complete output of free)? Have you run x264 in Windows with the same hardware and same amount of RAM?
I don't know how much, as it was all I could do to even get to force quit when it happened. I'll have to run another test to duplicate it. Are those to be done post-encode or do I have to have them active while x264 is running?

And yes, I've used x264 under Windows on both of the computers, as they're both dual-boot setups. XP Home 32-bit on both (mine is SP3, their's I think is still SP2, but I could be wrong).

[nm]

I don't know how much, as it was all I could do to even get to force quit when it happened. I'll have to run another test to duplicate it. Are those to be done post-encode or do I have to have them active while x264 is running?
ps, top and free only show current statistics, so you need to run them while encoding. If the system gets less and less responsive as the encoding progresses, there might be a memory leak somewhere. Follow x264's memory usage to find out.

How do you feed the input video to x264?

[qyot27]

I alternate between using raw I420 files (generated with mencoder) and piping it over from mplayer with yuv4mpeg. Under Windows I feed it an ffvhuff file using AviSynth, no filtering whatsoever.

[qyot27]

Took a few days before I had the time to run the tests (for curiousness' sake, I also tested under Fluxbox). Here were the results. These still used r1127*; I updated to r1129* this morning but haven't tested it. Nor have I tested the Athlon64 machine.

*the custom options+patches used:
--extra-cflags="-march=pentium3"
x264_win_zone_parse_fix_05.diff
x264_hrd_pulldown.10_interlace.diff

The command-line used:
x264 --crf 18.0 --ref 16 --mixed-refs --no-fast-pskip --bframes 16 --b-adapt 2 --b-pyramid --weightb --direct auto --deblock 1:1 --subme 9 --trellis 2 --partitions all --8x8dct --scenecut 100 --threads auto --thread-input --sar 1:1 --aud --progress --no-dct-decimate --no-psnr --no-ssim --output "output.mp4" "input.i420" 1280x720 --fps 59.94

Under GNOME:
x264 running
######@ubuntu-desktop:~$ free
total used free shared buffers cached
Mem: 253568 250060 3508 0 632 28412
-/+ buffers/cache: 221016 32552
Swap: 281096 206236 74860

x264 not running
######@ubuntu-desktop:~$ free
total used free shared buffers cached
Mem: 253568 85632 167936 0 272 20780
-/+ buffers/cache: 64580 188988
Swap: 281096 132456 148640

x264 running (top)
VIRT=518m
RES=160m
%CPU=5.8
%MEM=64.8

I noticed that the responsiveness got worse as the %CPU value dropped. It would start out high, and as encoding progressed, it would drop, and eventually plateau around the 3-6% range.

Under Fluxbox:
x264 running
######@ubuntu-desktop:/media/disk$ free
total used free shared buffers cached
Mem: 253568 250288 3280 0 232 19436
-/+ buffers/cache: 230620 22948
Swap: 281096 215516 65580

x264 not running
######@ubuntu-desktop:/media/disk$ free
total used free shared buffers cached
Mem: 253568 138780 114788 0 5996 57976
-/+ buffers/cache: 74808 178760
Swap: 281096 77328 203768
(Fluxbox generally rated 10m or so less under VIRT or RES than GNOME did, although I didn't let it plateau like I did on the other test; the %CPU and %MEM values I had to omit since I really didn't want to deal with a possible slowdown again)

[nm]

The command-line used:
x264 --crf 18.0 --ref 16 --mixed-refs --no-fast-pskip --bframes 16 --b-adapt 2 --b-pyramid --weightb --direct auto --deblock 1:1 --subme 9 --trellis 2 --partitions all --8x8dct --scenecut 100
--threads auto --thread-input --sar 1:1 --aud --progress --no-dct-decimate --no-psnr --no-ssim --output "output.mp4" "input.i420" 1280x720 --fps 59.94

[...]

I noticed that the responsiveness got worse as the %CPU value dropped. It would start out high, and as encoding progressed, it would drop, and eventually plateau around the 3-6% range.
You are running completely out of memory with that command-line. With 16 reference frames, b-adapt 2, 16 b-frames, and a 720p source, x264 needs over 420 MBs of resident memory. This leads to heavy swapping on a system with 256 MBs of RAM and you might run out of swap space too, which would mean that the encoding process gets terminated by the kernel. I tried both a 32-bit Windows build and a 64-bit Linux build of x264 and both had approximately the same memory usage.

The way how other running programs behave under these kind of extreme situations is determined by the process CPU and I/O priorities and how much the programs access disk and use RAM. I don't see how Windows would work better when a process runs out of memory and the system starts swapping.

To get decent performance out of x264, swapping must be avoided anyway. By using less references and b-frames (when using b-adapt 2), memory consumption is significantly lower (~100 MBs with --ref 5 --bframes 3), encoding is much faster and you don't really sacrifice any significant amount of quality.

[qyot27]

Ok, so it is simply running out of memory. Normally I don't encode 720p sources, but this was an odd case.

When you say 'over 420 MBs', is there any kind of set-in-stone limit for how much memory would be needed at a minimum for those options (with or without 720p as a deciding factor - as I said, I usually don't encode 720p, the cutoff is usually 720x480; or is the 518m that was reported by top said memory requirements?).

The reason being, according to the spec sheet for the i810e, I can put up to 512 MBs of RAM in; while I'm sure that even that still isn't enough, it would provide some level of relief, right? I realize how much of a case of diminishing returns it is to use that many b-frames and reference frames, but I designed that as an insane profile.

[nm]

When you say 'over 420 MBs', is there any kind of set-in-stone limit for how much memory would be needed at a minimum for those options (with or without 720p as a deciding factor - as I said, I usually don't encode 720p, the cutoff is usually 720x480; or is the 518m that was reported by top said memory requirements?).
Yes, memory consumption is determined by the algorithms used in x264 and the frame size. I haven't looked at the source code so I can't give a formula for calculating it. As a quick test, I just encoded with your settings on a machine that has more free memory than x264 required and checked the resident memory size after a while.

The reason being, according to the spec sheet for the i810e, I can put up to 512 MBs of RAM in; while I'm sure that even that still isn't enough, it would provide some level of relief, right? I realize how much of a case of diminishing returns it is to use that many b-frames and reference frames, but I designed that as an insane profile.
Well, if you want to encode HD video with such insane settings, the only solution is to get more RAM. 512 MBs may be enough for 720p if you don't use other programs at the same time, but for 1080p you would need to lower the number of references and b-frames.

[burfadel]

Yeah try something along the lines of 5 reference frames and 5 b-frames. Having 16 for both of these creates a massive speed loss, increase memory usage, and doesn't really provide much more benefit. Increasing the reference frames to 10 can be beneficial for animation but even then its usually only a few percent in those cases.