Error: malloc of size 228569

[ect5150]

Hey guys, didn't know if someone could point me in the right direction. Ever since I updated to version 2044 a while back I get this error message when encoding:

[Error] [8/13/2011 5:22:45 PM] An error occurred: x264 [error]: malloc of size 2285696 failed

Here is the command line out of MeGUI currently using 64-bit r2057:
program --preset placebo --pass 2 --bitrate 6500 --stats ".stats" --deblock -4:-4 --min-keyint 1 --ipratio 1.0 --pbratio 1.0 --ratetol 3.0 --qcomp 0.8 --rc-lookahead 250 --aq-strength 0.5 --merange 16 --psy-rd 1.0:0.25 --output "output" "input"

I know the settings are a little on the insane side, but I can deal with long encode times. I should note the error goes away if I disable the MB-Tree (--no-mbtree).

[Target Practice]

--rc-lookahead 250

you don't have enough ram for that

[ect5150]

--rc-lookahead 250

you don't have enough ram for that

I thought I had done this in the past, but maybe I had adjusted the settings then so it was not as RAM intense? I did decrease the value though and it seemed to work. Although the odd thing is I watched some system resource monitors and they claimed I had about 1Gig left of free RAM before it crash on me.

Thanks for the help though.

[LoRd_MuldeR]

If you are using a 32-Bit build of x264, then only 2 GB of RAM can be accessed. No matter how much physical RAM you have available!

The 2 GB per-process memory limit is a general x86 problem, not a x264-specific restriction.

With x64 (x86-64) this limitation has been resolved, but you will need a x64-capable CPU, a x64 operating system and a x64 executable.

[kemuri-_9]

LoRd_MuldeR,

are you just stating that to purely state it, or did you happen to miss reading
Here is the command line out of MeGUI currently using 64-bit r2057:?

[LoRd_MuldeR]

Yes, I missed that part. If he is using a 64-Bit build, the above comment doesn't apply here. But then getting malloc failures is strange :confused:

With 64-Bit builds, the virtual address space can't be the problem. And even if all physical RAM was used up, the OS should still be able to allocate more memory from the page file before failing.

Maybe some limitation of the C library?

[kemuri-_9]

Yes, I missed that part. If he is using a 64-Bit build, the above comment doesn't apply here. But then getting malloc failures is strange :confused:

With 64-Bit builds, the virtual address space can't be the problem. And even if all physical RAM was used up, the OS should still be able to allocate more memory from the page file before failing.

Maybe some limitation of the C library?

Yes, getting malloc failures is generally atypical for 64bit apps and it should be reconfirmed by the OP that he is indeed using a 64bit version.

If reconfirmed, then a possibility that comes to mind is that the pagefile could be disabled in combination with severe memory fragmentation so that there was no single available memory block that could handle the request for ~2.18MB of contiguous memory.
though this very unlikely...

[IanB]

In Avery Lee's recent blog, An amusing benefit of staying in 32-bit land (http://www.virtualdub.org/blog/pivot/entry.php?id=346#body), some of the responders note various means to constrain rogue 64bit processes from running amok. Shades of unix's ulimit.

[Mr Alpha]

Only Windows can allocate memory in the form of RAM or pagefile, and thus only Windows can fail to allocate such memory. Programs only access virtual memory and if a program fails to allocate memory then it is a failure to allocate virtual memory. Most likely he is hitting his commit limit.

[LoRd_MuldeR]

Only Windows can allocate memory in the form of RAM or pagefile, and thus only Windows can fail to allocate such memory. Programs only access virtual memory and if a program fails to allocate memory then it is a failure to allocate virtual memory. Most likely he is hitting his commit limit.

Not entirely true. If an application allocates heap memory via malloc(), then this memory is requested form the C library, because malloc() is a function of the C library. The implementation of malloc() depends on the individual C library, of course. But "internally" the C library will always use operating system functions (system calls) to request blocks of memory from the operating system. The memory blocks requested from the operating system will be mapped into the process' virtual address space. So if a malloc() fails, this means that all the heap memory currently mapped in the process' virtual address space is already assigned (or the free heap memory is too much fragmented to handle the request) and no further heap memory could be requested from the operating system. The latter can happen because: (1) The virtual address space is completely used up, so no more heap memory can be mapped to it. (2) The operating system is out of resources, i.e. all physical RAM and all page file is completely used up. (3) There are system policy restrictions that don't allow the user/process to allocate more memory. When running a 64-Bit process on a 64-Bit system, then (1) is extremely unlikely to happen...

See also:
http://en.wikipedia.org/wiki/Malloc#Implementations

[Mr Alpha]

Not entirely true. If an application allocates heap memory via malloc(), then this memory is requested form the C library, because malloc() is a function of the C library. The implementation of malloc() depends on the individual C library, of course. But "internally" the C library will always use operating system functions (system calls) to request blocks of memory from the operating system. The memory blocks requested from the operating system will be mapped into the process' virtual address space. So if a malloc() fails, this means that all the heap memory currently mapped in the process' virtual address space is already assigned (or the free heap memory is too much fragmented to handle the request) and no further heap memory could be requested from the operating system. The latter can happen because: (1) The virtual address space is completely used up, so no more heap memory can be mapped to it. (2) The operating system is out of resources, i.e. all physical RAM and all page file is completely used up. (3) There are system policy restrictions that don't allow the user/process to allocate more memory. When running a 64-Bit process on a 64-Bit system, then (1) is extremely unlikely to happen...

See also:
http://en.wikipedia.org/wiki/Malloc#ImplementationsTrue, but the memory allocated by the process is still virtual memory.

As you say, with a 64-bit process (1) is extremely unlikely.

Due to the way Windows handles virtual memory (2) is also extremely unlikely. Windows won't commit more virtual memory than RAM + pagefile, and virtual memory usage is always greater than actual memory usage (due to shared memory mapped images of system dlls if nothing else). On the other hand your actual memory limit is always greater than your virtual memory limit because files mapped into memory from the hard drive aren't backed by the pagefile. So your actual memory limit is RAM + pagefile + files backed by the hard drive but not your pagefile. This means you will run out of virtual memory before you fill up RAM + pagefile.

(3) is always, of course, a real possibility.

I would still add a (4) The system hit the commit limit and Windows refuses the request for more memory.

[bxyhxyh]

If i repair or reinstall OS, will it be fixed?