Error encoding 3840x2160/60p video

[mariner]

Greetings.

Need help encoding 150mbps 3940x2160/59.94p h264 video, starting from a Panasonic 24mbps 1920x1080/59.94p clip.

I tried the following, both ended with errors:

1. Using Lanczos resize:

FFVideoSource("Ginza_cat.m2ts")
LanczosResize(3840,2160)
ConverttoYv12
AssumeFPS(60000,1001)

error message:
E:\Clips\4k ginza>x264 -o gz.mkv gz.avs
avs [info]: 3840x2160p 0:0 @ 60000/1001 fps (cfr)
x264 [warning]: MB rate (1942057) > level limit (983040)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 FastShuffle SSE4.1 Cach
e64
x264 [info]: profile High, level 5.1
x264 [error]: malloc of size 8682624 failed
x264 [error]: x264_encoder_encode failed
aborted at input frame 47, output frame 0

2. Using Horizontal and Vertical Stacking:

v=FFVideoSource("Ginza_cat.m2ts")
Vh = StackHorizontal(v,v)
vv = StackVertical(vh,vh)
ConverttoYv12(vv)
AssumeFPS(60000,1001)

Error message:
E:\Clips\4k ginza>x264 -o gzhv.mkv gzhv.avs
avs [info]: 3840x2160p 0:0 @ 60000/1001 fps (cfr)
x264 [warning]: MB rate (1942057) > level limit (983040)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 FastShuffle SSE4.1 Cach
e64
x264 [info]: profile High, level 5.1
x264 [error]: malloc of size 8682624 failed
x264 [error]: x264_encoder_encode failed
avs [error]: GetFrameBuffer: Returned a VFB with a 0 data pointer!
size=12441664, max=536870912, used=208398528
I think we have run out of memory folks! occurred while reading frame 46
aborted at input frame 46, output frame 0

System: W7x86/4GB Ram

Appreciate your kind assistance.

Many thanks and best regards.

[smok3]

i'd try to skip avisynth and do the thingy using ffmpeg or x264 alone (I had no problems with 4k footage, downscaling to fullHD and encoding to x264 with ffmpeg - on os x with 3gigs of ram)

[nm]

i'd try to skip avisynth and do the thingy using ffmpeg or x264 alone

Good idea.

(I had no problems with 4k footage, downscaling to fullHD and encoding to x264 with ffmpeg - on os x with 3gigs of ram)

But mariner is going the other way. Memory consumption of 4k encoding is much higher than 1080p encoding, and x264's lookahead needs quite a lot of RAM. Decoder doesn't consume too much in either case, but AviSynth might, so having SetMemoryMax(256) in the script could help too.

Overall I'd expect mariner's x264 process to allocate almost 4 times the memory you needed (if you both used default settings).

[Dark Shikari]

Use a smaller setmemorymax() in Avisynth, and/or use less memory-intensive x264 settings, and/or use 64-bit x264.

[smok3]

But mariner is going the other way. Memory consumption of 4k encoding is much higher than 1080p encoding

just tested with 4256 × 2832 (via ffmpeg) and the system came to crawl (can't move the mouse, but it might probably encode something if i had month for the task ....), so thats true :)

x264 lib was using:
-preset medium -tune film -crf 21 -threads 0

[Juce]

I had the same problem with 32 bit Windows XP. With these settings I got it to work:
--rc-lookahead 40 -b 4 --preset veryslow --ref 16

"--preset veryslow --ref 16" is unnecessary.

[Blue_MiSfit]

I'd imagine 64 bit is basically mandatory to do any work with 4k material...

[mariner]

i'd try to skip avisynth and do the thingy using ffmpeg or x264 alone (I had no problems with 4k footage, downscaling to fullHD and encoding to x264 with ffmpeg - on os x with 3gigs of ram)

Thanks for the kind reply, smok3.

Tried this, but failed:

E:\Clips\4k ginza>x264 --fps 60000/1001 --bitrate 150000 --video-filter resize:w
idth=3840,height=2160,method=fastbilinear -o 4kx264.mkv Ginza_cat.m2ts

ffms [info]: 1920x1080p 1:1 @ 60001/1001 fps (cfr)
resize [info]: resizing to 3840x2160
x264 [info]: using SAR=1/1
x264 [warning]: MB rate (1942057) > level limit (983040)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 FastShuffle SSE4.1 Cach
e64
x264 [info]: profile High, level 5.1
x264 [error]: malloc of size 9078784 failed
x264 [error]: x264_encoder_encode failed
x264 [info]: final ratefactor: 2.07

aborted at input frame 50, output frame 0

Got this if --bitrate removed:
E:\Clips\4k ginza>x264 --fps 60000/1001 --video-filter resize:width=3840,height=
2160,method=fastbilinear -o 4kx264.mkv Ginza_cat.m2ts

ffms [info]: 1920x1080p 1:1 @ 60001/1001 fps (cfr)
resize [info]: resizing to 3840x2160
x264 [info]: using SAR=1/1
x264 [warning]: MB rate (1942057) > level limit (983040)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 FastShuffle SSE4.1 Cach
e64
x264 [info]: profile High, level 5.1
x264 [error]: malloc of size 34730496 faileds, eta 1:35:12
x264 [error]: x264_encoder_encode failed
x264 [info]: frame I:1 Avg QP:27.72 size:262223
x264 [info]: frame P:3 Avg QP:28.17 size:106464
x264 [info]: frame B:1 Avg QP:31.16 size: 25352
x264 [info]: consecutive B-frames: 60.0% 40.0% 0.0% 0.0%
x264 [info]: mb I I16..4: 15.3% 77.7% 7.0%
x264 [info]: mb P I16..4: 4.8% 22.4% 1.1% P16..4: 36.3% 4.8% 1.9% 0.0% 0
.0% skip:28.6%
x264 [info]: mb B I16..4: 0.1% 0.0% 0.0% B16..8: 33.9% 0.5% 0.0% direct:
0.8% skip:64.6% L0:23.9% L1:76.0% BI: 0.1%
x264 [info]: 8x8 transform intra:78.4% inter:92.4%
x264 [info]: coded y,uvDC,uvAC intra: 64.0% 50.6% 8.0% inter: 14.7% 12.2% 0.0%
x264 [info]: i16 v,h,dc,p: 14% 26% 7% 54%
x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 21% 14% 14% 7% 9% 10% 7% 10% 8%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 28% 12% 8% 7% 13% 13% 7% 8% 4%
x264 [info]: i8c dc,h,v,p: 57% 23% 16% 4%
x264 [info]: Weighted P-Frames: Y:0.0% UV:0.0%
x264 [info]: ref P L0: 74.1% 13.4% 11.1% 1.4%
x264 [info]: kb/s:58210.72

aborted at input frame 55, output frame 5
encoded 5 frames, 0.53 fps, 72847.43 kb/s

What do I do next?

[mariner]

Good idea.

But mariner is going the other way. Memory consumption of 4k encoding is much higher than 1080p encoding, and x264's lookahead needs quite a lot of RAM. Decoder doesn't consume too much in either case, but AviSynth might, so having SetMemoryMax(256) in the script could help too.

Overall I'd expect mariner's x264 process to allocate almost 4 times the memory you needed (if you both used default settings).

Use a smaller setmemorymax() in Avisynth, and/or use less memory-intensive x264 settings, and/or use 64-bit x264.


Thansk for the kind reply, nm and Dark Shikari.

Added SetMemoryMax(256) but same error, it seems:

FFVideoSource("Ginza_cat.m2ts")
SetMemoryMax(256)
LanczosResize(3840,2160)
ConverttoYv12
AssumeFPS(60000,1001)

E:\Clips\4k ginza>x264 -o gz.mkv gz.avs

avs [info]: 3840x2160p 0:0 @ 60000/1001 fps (cfr)
x264 [warning]: MB rate (1942057) > level limit (983040)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 FastShuffle SSE4.1 Cach
e64
x264 [info]: profile High, level 5.1
x264 [error]: malloc of size 8682624 failed
x264 [error]: x264_encoder_encode failed

aborted at input frame 47, output frame 0

What do I do next?

[sneaker_ger]

Use a pipe and lower --rc-lookahead to e.g. 25 for starters:
avs2pipemod -y4mp gz.avs | x264 - --demuxer y4m -o gz.mkv --rc-lookahead 25

RAM is dirt cheap at the moment, btw. Good time to upgrade RAM and go for a 64 bit OS.

[smok3]

as said, less memory intensive settings, maybe "--preset ultrafast --tune fastdecode" would fit.

[nm]

as said, less memory intensive settings, maybe "--preset ultrafast --tune fastdecode" would fit.

Ultrafast should be avoided at all cost. Lower rc-lookahead will do it even when using slow presets.

[Blue_MiSfit]

If you must stick with (32 bit) avisynth, then consider using avs2avi to pipe into 64 bit x264.

[sneaker_ger]

He's on 32 bit Windows.

[Blue_MiSfit]

Sorry for missing that detail. That would indeed prevent using 64 bit x264!! :)

Derek

[mariner]

Use a pipe and lower --rc-lookahead to e.g. 25 for starters:
avs2pipemod -y4mp gz.avs | x264 - --demuxer y4m -o gz.mkv --rc-lookahead 25

RAM is dirt cheap at the moment, btw. Good time to upgrade RAM and go for a 64 bit OS.

Thanks for the kind reply, sneaker_ger. Works brilliantly.

1. The number of lookahead frames has to be lowered to 10 when using horizontal and vertical stacking. How much does this compromise the encoding quality?

2. Any free storage sites that keep the uploads for a fairly long period of time?

Many thanks and best regards.

[mariner]

I had the same problem with 32 bit Windows XP. With these settings I got it to work:
--rc-lookahead 40 -b 4 --preset veryslow --ref 16

"--preset veryslow --ref 16" is unnecessary.

It seems the key to the solution has been in plain sight all this time... Many thanks for the suggestions, Juce.

[mariner]

Having difficulties with multiupload. Are there alternatives for hosting 1GB file?

Quick testing with Sapphire HD6670/Q8400/W7x86:

1. Potplayer: reverts to software decoding.
2. Arcsoft decoder: Half the screen is green.
3. FFdshow: Q8400 can only manage about 20+ fps.

So it seems UVD3 fails when decoding 3840x2160 video in DXVA mode.

For software decoding, something significantly more powerful than Q8400 is required.

http://img21.imageshack.us/img21/7222/4kqhdc.jpg (http://imageshack.us/photo/my-images/21/4kqhdc.jpg/)

[CruNcher]

you should though anyways confirm this with another encoder bitstream just to be sure

[mariner]

https://rapidshare.com/files/1049894694/4k_QHD_Cats_3840x2160x60p_150mbps_.mp4

[mariner]

4k_QHD_Cats_3840x2160x60p_150mbps .mp4 (http://depositfiles.com/files/he0rq52j8)

The clip is now on depositfiles.

VP5 has been reported to handle low bit rate 4k video. It would be interesting to see how it copes with this.

Not sure about SB and UVD3+(7 series).

It would be nice to see some benchmarks with these cards.

[mariner]

Some disappointing news.

It was reported that both SB and 7xxx do not support 4k hardware decoding.

Will return the H67s MB and stay with the Sapphire 6670 Ultra/G31 setup. Hopefully the new IVB and 8xxx will support HDMI 1.5 as well.

Anyone with a GT545 to test these clips?

[mariner]

JVC Europe has posted 4K testfile taken by the GY-HMQ10:

ftp://publicusr:readpublic~@ftp.jvcpro.eu/Downloads/4KTestfiles_short_Version.zip

It is made up of four 1920x1080x60p 34mbps FHD clips. Does anyone know how to combine these in one 4k QHD video without re-encoding?

Many thanks and best regards.

[kemuri-_9]

It is made up of four 1920x1080x60p 34mbps FHD clips. Does anyone know how to combine these in one 4k QHD video without re-encoding?

this would require magic pixie dust, which is hard to get a hold of these days.

without it, re-encoding is required.

[mariner]

4KClipmanagerソフトウェア | JVC

http://www3.jvckenwood.com/pro/video/gy-hmq10/4k-clipmanager.html

http://www3.jvckenwood.com/pro/video/gy-hmq10/dl/data/JVC_4K_Clip_Manager.zip

[mariner]

4k sakura clip
4096x2160x60p @130mbps
http://av.watch.impress.co.jp/video/avw/docs/525/446/smple_4k.zip

More raw 34mbps HD clip for each quadrant.
http://av.watch.impress.co.jp/video/avw/docs/525/446/13a_0027.mp4
http://av.watch.impress.co.jp/video/avw/docs/525/446/13b_0027.mp4
http://av.watch.impress.co.jp/video/avw/docs/525/446/13c_0027.mp4
http://av.watch.impress.co.jp/video/avw/docs/525/446/13d_0027.mp4

[kieranrk]

this would require magic pixie dust, which is hard to get a hold of these days.

without it, re-encoding is required.

Theoretically possible with FMO I think.

[mariner]

Windows version of 4KClipmanagerソフトウェア | JVC. (http://www3.jvckenwood.com/pro/soft_dl/4k-clipmanager/data/JVC_4K_Clip_Manager_Windows.zip)

Did anyone figure out how to get it to work?

Here's the delicious 4K 30fps Roast Duck (http://depositfiles.com/files/3usxranbg) clip taken by a Red.