Is x264 CLI some way filesystem limited?

[Anakunda]

Hello, got a strange crash today during BR conversion,
x264 terminated suddenly with error code, this is an error message

x264 encoding failed

What is strange is that x264 terminated preliminary with usual report saying nothing about error, just returning positive error code.
I noticed that the incomplete stream was just almost a byte exactly 2GB in size, so interested if it could have some problem with 2GB limitation....but the drive was NTFS so I think should be no problem....

[LoRd_MuldeR]

If there really is some problem with "big" files, then it's most likely not a limitation of the file system itself, but a limitation of the API used to access the file system.

x264 is written in C. And the standard C file functions, such as fseek() and ftell(), are usually defined with 32-Bit data types (at least on 32-Bit system), so they cannot look beyond the 4 GB barrier.

There exists improved versions of seek(), ftell() and friends that are guaranteed to be defined with 64-Bit data types (even on 32-Bit system), but these are compiler-/platform-specific :rolleyes:

(For example on MSVC you need to use _fseeki64(), but on GCC you have to use fseeko() and pass '-D_FILE_OFFSET_BITS=64' to the compiler instead)

[Anakunda]

So using 32bit x264 is likely to crash on outputs >2GB?
I think 32bit signed integer only can address -2^31...2^31-1 which is pretty few.
Not sure if using avs4x264 as frontend to 64bit x264 does avoid this problem.

[LoRd_MuldeR]

So using 32bit x264 is likely to crash on outputs >2GB?

Well, there might be a problem of this kind hiding in one of x264' muxers. But your problem might be something completely different!

We just have too few information to tell. And to make it clear again: 32-Bit applications can access files larger than 4 GB perfectly fine - if written properly.

I think 32bit signed integer only can address -2^31...2^31-1 which is pretty few.

Yes, when using signed 32-Bit integers as file offsets (and fseek() uses a signed type, because it can seek in both directions), you are limited to 2 GB.

With unsigned 32-Bit integers you can address 4 GB of data, at least.

Not sure if using avs4x264 as frontend to 64bit x264 does avoid this problem.

You can try a 64-Bit build of x264 and find out yourself...

[MasterNobody]

Correctly compiled x264 doesn't have problems with large files (of course if you don't use FAT32 with its limitations). GCC compiled builds use fseeko64/ftello64 and ICL compiled use _fseeki64/_ftelli64. So your problem most probably something other. Also there is no such message as "x264 encoding failed" in x264. May be you mean "x264_encoder_encode failed" but x264 usually write more informative reason before this message. And next time provide error message exactly as it was showed or preferably full copy of console log (not only one last message).

[Blue_MiSfit]

I regularly generate very large outputs with x264 (14+ GB), so there's nothing about x264 in particular that causes this. This looks like a compilation error to me...

Derek