Avisynth issues when processing HD content?

[JeanMarc]

I am experimenting with HD content and 64-bit applications. My attempts to process a 1920x1080 m2ts clip with 32-bit applications generally failed. What I was trying to do, to get started, is load the m2ts file into an avisynth script, process and resize it to 1280x720 in avisynth, encode it into x264 mp4 or mkv with ffmpeg.
Here is the kind of script I am starting with:
Loadplugin("C:\Program Files\Avisynth64_100416\ffms-2.17-x64\ffms2-x64.dll")
a = FFAudioSource("E:\Barracuda1\_Videos_shared\_videos to do\m2ts-test\00006.m2ts")
v = FFVideoSource("E:\Barracuda1\_Videos_shared\_videos to do\m2ts-test\00006.m2ts", \
fpsnum = 30000, fpsden = 1001, width = 1280, height = 720, resizer = "SPLINE")
audiodub(v, a)
With 32-bit applications, it looks like I always hit a 2GB memory limitation , causing the application to use more and more memory, and to crash before the job is completed, and I see that pattern happening with any avisynth 32-bit script and HD content, even with the most basic avs script – but never with “standard” SD mpeg-2 content.

Encoding with ffmpeg64 and avisynth64 is working well for me (I used Avisynth 2.58 tsp MT version5(mod seraphy), built Mar 19 2010) and a very recent 64-bit ffmpeg built from Zeranoe.

But I usually want to do more with avisynth, for example, test the quality of the video with the Avisynth SSIM plugin. I haven't found any 64-bit version of SSIM, and any attempt to use SSIM with 32-bit avisynth **and** HD material (it works fine on 720x480 content) makes the application crash after processing a number of frames.
I tried running the avisynth script containing the SSIM function with VirtualDub, MPC-HC, PotPlayer, even avfs (where I converted the avs script for the original video and the script for the compressed video into virtual avi file before running just the SSIM plugin with 32-bit avisynth.) All crash after a while before completion. I see on Windows Task Manager the amount of memory used by the process continually increasing as the number of frames processed increase, until (apparently when hitting around 1,900,000K) it crashes.
One solution I see is to cut the video into small pieces, and process it one piece at a time. Possible, but not nice. I also understand there are ways to pipe processes between 32-bit and 64-bit, but I have never tried it yet and I am not sure how that would resolve my problem.
My conclusion is that (1) 64-bit avisynth and ffmpeg (and Virtualdub and MPC-HC) work fine if you stick to the basics but not as soon as you need specific 32-bit plugins, and (2) regular 32-bit avisynth presents memory problems with HD content.

So the more specific questions are:

Am I right in assuming that when you deal with HD content, 32-bit avisynth has memory problems leading to crashes?
If that is true, is there a workaround, or a solution in sight?
The only HD content I have is h264 / m2ts. Could the problem be related to loading that kind of video into avisynth using ffmpegSource?
Do I miss anything obvious?

Thanks for any feedback

[Didée]

With 32-bit applications, it looks like I always hit a 2GB memory limitation , causing the application to use more and more memory, and to crash before the job is completed,
That should not be. And many-many people, including myself, are regularly processing Full-HD content in 32bit Avisynth without any problem.

Have you tried to explicitely specify a SetMemoryMax() at the start of the script? Some of the 32bit MT-builds do not impose the standard framecache limitations, (consider it a bug), and will indeed crash very soon, when SetMemoryMax isn't specified by the user. Set a MemoryMax(512) or (1024) or whatever fits the job, and everything runs fine and stable.

[JeanMarc]

Thanks for the advice. I already tried it, but checked again. I added SetMemoryMax() at the beginning of the script.
Without SetMemoryMax, VirtualDub crashes at frame #23983
With SetMemoryMax(512), crash at frame #23716
With SetMemoryMax(1024), crash at fame #24755
No significant difference. The whole video has 185094 frames.
The error message is of the type:
Avisynth read error:
Avisynth access violation at 0x0003A65C in C:\Program Files (x86)...\ffms2.dll,
attempting to write to 0x000075F9
And the error always appear when the memory used by VirtualDub, which is always increasing with each processed frame, reaches around 1,900,000K. It doesn't seem to matter whether I am running the script with VirtualDub or another application.

I thought it was an HD problem, as I am using pretty much the same tools used for SD, where they worked flawlessly.
The only difference I see is loading the m2ts file with FFAudioSource and FFVideoSource, which I was using only with h264 / mp4 file with SD content.
I am still in the dark!

[JeanMarc]

I indeed just verified that the problem is not with all HD content: I replaced my m2ts file with an HD h264 mp4 file I had generated with approximately the same frame size.
Two major differences: it runs faster, and the process memory usage stays around 325,000K instead of increasing forever.
My conclusion: the m2ts file I am using to validate my tools has some problems. I will look at it more closely.

Thanks for any advice regarding m2ts specific issues.

[Didée]

Sound like a memory leak in the source filter, i.e. ffms2.dll. Not really a fault of "Avisynth 32bit". The SetMemoryMax value controls only the frame cache of the Avisynth core. It's not responsible for a plugin's internal memory managment.

ffms2 is a handy thing when it happens to work. But there's always this "there be dragons" warning about it.

[JeanMarc]

I am using the same ffms2.dll in both cases. I guess you mean ffms2 can behave differently depending on the type of content?

[naoan]

Sounds like problem with virtualdub not being large address aware, see http://forum.doom9.org/showthread.php?p=1607607#post1607607

[JeanMarc]

I am trying the ffmpeg lossless conversion to avi with huffyuv as an intermediate step. I used ffmpeg -i source.m2ts -threads 8 -c:v huffyuv -an lossless6.avi
It's working, but somewhat slower than what I expected (< 30fps with i7-3770K), and only 4 threads working. It doesn't matter if I specify -threads 0 or -threads 8. Real slow for something that is not even compressed!
I may try other codecs.
Any recommendation?

[kolak]

It can do even 500fps on i7 3.5GHZ.
I done some HD decoding benchmarks and on 12 cores system it can do 1200fps with 1.5GByte/sec reading :) (you need a proper RAID to do it)
Speed and multithreading is very good, but be aware than ffmpeg does encode it in single thread (at leas version 1).

[StainlessS]

I use Ut Video and like it but JFYI, there is a multi-thread version of HuffYUV on MediaFire in my sig
(together with standard version 2.1.1), although I dont see much difference on my dual core.
It uses a different FOURCC and must be decoded with same decoder (EDIT and so will not play in VLC).
Just had another look at it, also contains 64 bit version and another version with CCESP patch 0.25,
and source code for all versions. Originated from a Japanese site (I think) some English instructions
added by someone.

[JeanMarc]

I just compared utvideo with huffyuv: 42 fps vs 27 fps. Nice improvement, especially with the utvideo avi file being half the size of the huffyuv file :). This is on i7 3700 MHz. My source is YUV, 1920x1080.

@kolak, Curious about how you achieve 500 fps?
Or what can I do to improve performance (other than overclocking which I haven't tried yet, but will soon).

[kolak]

My numbers are for pure decoding- no other processing. I was testing just speed of the codec itself. It does 1200fps on 12 core system for HD res.
ffmpeg implementation is not multithreaded as far as I know.

[JeanMarc]

On these tests I don't do any processing either:
ffmpeg -i source.m2ts -frames:v 6000 -threads 0 -c:v utvideo -an losslessUt.avi
But I see only 4 threads working -- not very hard -- cpu utilization ~ 5%.

[kolak]

5% CPU only? Try to put -threads before -i (is 0=number of cores?, maybe try 4).

Dont use ffmpeg. You can try to use some GPU based AVC decoder (maybe LAV decoder) and Vdub for encoding with latest UTVideo codec. UtVideo in ffmpeg is not multithreaded, but even so it should do 50fps on 1 core.

[kolak]

Y range lost? What does it mean?
You can set working color space as you want to avoid any further conversions.

[JeanMarc]

5% CPU only? Try to put -threads before -i (is 0=number of cores?, maybe try 4). -threads 0 is supposed to let the software pick the right number of cores. Putting -threads before -i didn't help, it slowed it down a little.

Now I noticed that if I do shorter clip (6000 frames) I get a fairly fast encoding ~300 fps. But with a longer video, the fps keeps declining.

Regarding GPU-based codecs, I have the integrated graphics. Not sure if I would get an improvement.
But I need to try vdub. I noticed that it is often faster than ffmpeg. But I want to keep using the command line and I haven't totally explored how to tell vdub what configuration to use in command line (I think I have seen it done)