Encode UHD prores 444(4)

[anton_foy]

Hi, I am trying to encode a UHD (3840x2160) video filtered with Avisynth+ and encode prores444 (or 4444) but the only thing I can get is the Prores as "YUV 8-bit 4:2:0" to work and the needed info gets lost.

This is how it looks in "External encoder";
-> "Edit external encoder profile", Command Arguments: -f rawvideo -s %(width)x%(height) -r %(fpsnum)/%(fpsden) -i - -c:v prores_ks -profile:v 4 -pix_fmt yuv444p10 "%(tempvideofile)"
Under the "video"-tab I want to use "yuv444p16le - 16-bit YUV 4:4:4" but this just makes VDUB2 crash instantly when trying to encode.
Error: Oops -- VirtualDub has crashed... Or out of memory error.

Just for test I have tried "yuv444p - 8-bit YUV 4:4:4", this gives me a green-blueish tinted image and "yuv422p - 8-bit YUV 4:2:2"
gives me a green-tinted image.

Is it even possible to encode in higher than 8-bit and 4:4:4 prores (or DNXHR) in VirtualDub2 (Build 44282)?

Thanks

[poisondeathray]

Don't use the external encoder.

Use the vdub2 FFMpeg / Apple ProRes option under video=>compression

[shekh]

This should work, no idea the for the reason of crash. Do you run x64 version? Otherwise low memory could be the issue.

For the command line you have to also put pix_fmt of the rawvideo:

example ffv1
-f rawvideo -pix_fmt %(pix_fmt) -s %(width)x%(height) -r %(fpsnum)/%(fpsden) -i - -c:v ffv1 %(tempvideofile)

prores
-f rawvideo -pix_fmt %(pix_fmt) -s %(width)x%(height) -r %(fpsnum)/%(fpsden) -i - -c:v prores_ks -profile:v 4 -pix_fmt yuv444p10 "%(tempvideofile)"

[poisondeathray]

Or why not use ffmpeg directly with avs input ? What is the reason for the intermediate vdub2 step ?

[anton_foy]

Thank you both.

@poisondeathray
When I use the vdub2 FFMpeg / Apple ProRes option under video=>compression and "save video" as video.mov I get memory error "Out of memory (unable to allocate 66359312 bytes)". Too old computer?

I would love to use ffmpeg directly with avs input but all I get is error when I try it: "ffmpeg outputfile #0 does not contain any stream" and it doesn't write any frames.
ffmpeg prorestest.avs prores_ks -c:v -profile:v 4 -pix_fmt yuv444p10 test.mov

@shekh
I used your line -f rawvideo -pix_fmt %(pix_fmt) -s %(width)x%(height) -r %(fpsnum)/%(fpsden) -i - -c:v prores_ks -profile:v 4 -pix_fmt yuv444p10 "%(tempvideofile)" But I get "Out of memory (unable to allocate 49766415 bytes)".

[shekh]

If you have to run 32-bit, maybe look at Option->Performance->Video buffering (change to minimum 4)
Not sure what else could help to save memory.

[anton_foy]

If you have to run 32-bit, maybe look at Option->Performance->Video buffering (change to minimum 4)
Not sure what else could help to save memory.

Thanks but I get the same memory error. It does not even letting me export as png-sequence...
If I edit my avs-file to 8-bit I have no problem but as soon as I change it to deep color it refuses.

[poisondeathray]

I would love to use ffmpeg directly with avs input but all I get is error when I try it: "ffmpeg outputfile #0 does not contain any stream" and it doesn't write any frames.
ffmpeg prorestest.avs prores_ks -c:v -profile:v 4 -pix_fmt yuv444p10 test.mov



Typos ?

It should be like this

ffmpeg -i prorestest.avs -c:v prores_ks -profile:v 4 -pix_fmt yuv444p10 test.mov

[anton_foy]

@poisondeathray
Thanks, I probably do it wrong somehow but I have the avs-script in the same folder as ffmpeg.exe and in the Command Prompt I pasted your line but I now get error:
prorestest.avs: unknown error occured I found another person here in the forums getting this and he renamed ffmpeg.exe to ffmpeg32.exe to make it work but it does not work for me.

[poisondeathray]

Your avisynth build has to match the ffmpeg binary in terms of x86 vs. x64 . eg. If you had avisynth x86 installed, you'd need a ffmpeg x86 binary

[anton_foy]

@poisondeathray
Thank you now it works :) !!! I had dragged and dropped the exe-file in the wrong place, the old x64 was still there and I mixed them up.
But should it say below input: Stream #0:0 Video: rawvideo (Y3[11][16] / 0x100B3359), yuv420p16le...?
The encoding section shows yuv444p10le which is fine I guess ( because FFMPEG cannot encode prores 4444 in higher than 10 bit?)

[poisondeathray]

all ffmpeg prores encoders currently only support 10bit (prores 4444 and 4444xq are supposed to be 12bit) ; this is well documented and requests have been made to extend proper 12bit support

[poisondeathray]

But should it say below input: Stream #0:0 Video: rawvideo (Y3[11][16] / 0x100B3359), yuv420p16le...?
The encoding section shows yuv444p10le which is fine I guess ( because FFMPEG cannot encode prores 4444 in higher than 10 bit?)

What are you sending in the avs script ? what does info() say ?

It should be yuv444p10le, not yuv420p16le . Otherwise you are letting ffmpeg downconvert the bit depth and upsample the chroma, instead of specifying how you want it to be done. e.g. what algorithm are you using to upscale ? Did you want to dither down ? What algorithm to dither (or not)


Stream #0:0: Video: rawvideo (Y3[0][10] / 0xA003359), yuv444p10le, 3840x2160
, 24 fps, 24 tbr, 24 tbn, 24 tbc

[Blue_MiSfit]

10 bit is perfectly valid for 4444 and 4444 XQ, they both just happen to support 10 and 12 bit encoding ;)

[anton_foy]

What are you sending in the avs script ? what does info() say ?

It should be yuv444p10le, not yuv420p16le . Otherwise you are letting ffmpeg downconvert the bit depth and upsample the chroma, instead of specifying how you want it to be done. e.g. what algorithm are you using to upscale ? Did you want to dither down ? What algorithm to dither (or not)


Stream #0:0: Video: rawvideo (Y3[0][10] / 0xA003359), yuv444p10le, 3840x2160
, 24 fps, 24 tbr, 24 tbn, 24 tbc


My avs-script ends with:

ConverttoYUV444()
Convertbits(10, dither=1)

I can post the full script when I come home from work.

EDIT:
Here is my script
### Avisynth+ 3.6.1 (r3300, 3.6, I386) ###
SetFilterMTMode("DEFAULT_MT_MODE", 2)

#Index
LSMASHVideoSource("F:\sony\C1031.mp4")
trim(17,24)

convertbits(16)
Levels(0, 1, 65280, 0, 65280, coring=false, dither=false)

### DENOISE ###
neo_dfttest(sigma=16,sigma2=16,f0beta=1.0,tbsize=1,sbsize=120,sosize=90, y=2 , u=3, v=3) #sbsize=120,sosize=90 / sbsize=24,sosize=18
mergeluma(TemporalSoften(3, 4, 8, mode=2))

### TOPSHARP ###
shrp=sharpen(1.0)
vid1 = last
shine1 = vid1.Overlay(shrp, mode="blend", opacity=1.0)
emask1 = vid1.mt_edge(mode="prewitt",thy1=0,thy2=65280).mt_expand().fastgaussblur(10)
result1 = mt_merge(vid1,shine1,emask1, U=2,V=2)
merge(result1)

### SHINE ###
blr=mt_expand().Blur(1.56).Blur(1.56).Blur(1.56).Blur(1.56).Blur(1.56).Blur(1.56).Blur(1.56) #BinomialBlur(varY=4.5, varC=6.5, Y=3, U=2, V=2, useMMX=true)
vid = last
shine = vid.Overlay(blr, mode="blend", opacity=0.9)
emask = vid.mt_edge(mode="prewitt",thy1=0,thy2=65280).mt_expand().mt_expand().mt_expand().fastgaussblur(20)
result = mt_merge(vid,shine,emask, U=2,V=2)
return(result)

#Debanding
neo_f3kdb(range=15, Y=45, Cb=30, Cr=30, grainY=0, grainC=0, sample_mode=2, blur_first=true, dynamic_grain=false, mt=true)

#Chroma Upscaling
ConverttoYUV444()

Prefetch(4)
ConvertBits(bits=10, dither=1)

[anton_foy]

Could the MT-mode interfere with the last part: Converttoyuv444() and Convertbits(bits=10, dither=1)
or is there another reason why FFMPEG interprets my input as "yuv420p16le" instead of "yuv444p10le"?

Thanks

[Cary Knoop]

What's the original chroma-subsampling and bit-depth?

Could you show a few seconds of pre and post processing.

[anton_foy]

What's the original chroma-subsampling and bit-depth?

Could you show a few seconds of pre and post processing.


@Cary Knoop
Its footage from my Sony A6300 camera.
XAVC S: 3840 x 2160p / 25 fps (100 Mbps), 4:2:0, 8 bit.

As seen in my avs-script I filter it in 16-bits and then convert into 10-bit yuv444.

Yes I can show before and after clip as soon as I come home from work.
I have imported the clip in Davinci Resolve and compared it to avisynth filtered in 8bit and the new one is indeed in 10 bits. I am just concerned that ffmpeg does not interpret it as yuv444 10-bit input clip which it should.

[anton_foy]

Here is the link with the videos, processed and unprocessed: https://we.tl/t-yOeb6Tolr4

[poisondeathray]

Maybe a problem with your ffmpeg binary. What does info() report ?



Blankclip(pixel_type="YV12").KillAudio()
convertbits(16)
ConverttoYUV444()
Prefetch(4)
ConvertBits(bits=10, dither=1)


For this test script, info() reports YUV444P10 ; and ffmpeg -i input.avs reports yuv444p10le

Why don't you have ConvertBits before prefetch ?

[anton_foy]

@poisondeathray
Maybe a problem with your ffmpeg binary. What does info() report ?
Sorry but how do I use "info()"? Im not familiar with this command.


Code:

Blankclip(pixel_type="YV12").KillAudio()
convertbits(16)
ConverttoYUV444()
Prefetch(4)
ConvertBits(bits=10, dither=1)

For this test script, info() reports YUV444P10 ; and ffmpeg -i input.avs reports yuv444p10le

This is what my ffmpeg reports:
Input #0, avisynth, from 'input.avs':
Duration: 00:00:10.00, start: 0.000000, bitrate: 0 kb/s
Stream #0:0: Video: rawvideo (Y3[0][10] / 0xA003359), yuv444p10le, 640x480, 24 fps, 24 tbr, 24 tbn, 24 tbc
At least one output file must be specified

Why don't you have ConvertBits before prefetch ?
I was given this advice from another member in doom9 in the Avisynth section. But I tried to put prefetch(4) last and it did not make a difference.

[poisondeathray]

The problem is your script. It's returning YUV420P16

The problem line is

return(result)


Instead, just call result


#return(result)
result


That "result" gets passed into the rest of the script neo_f3kdb, etc... as "implied last"

[poisondeathray]

@poisondeathray

Sorry but how do I use "info()"? Im not familiar with this command.



When someone asks for info(), it refers to the avisynth script. You just add info() to the very end. It tells you what the script characteristics are, including colorspace(pixel format). In your case it was yuv420p16 , so that was what ffmpeg was being "fed"


Blankclip(pixel_type="YV12").KillAudio()
convertbits(16)
ConverttoYUV444()
Prefetch(4)
ConvertBits(bits=10, dither=1)
info()


On the other test script, it says YUV444P10

[anton_foy]

@poisondeathray
Aha many thanks!!! I will try this in the morning and let you know how it worked :)
Back to the job again in a few hours...

[age]

This should work, no idea the for the reason of crash. Do you run x64 version? Otherwise low memory could be the issue.

For the command line you have to also put pix_fmt of the rawvideo:

example ffv1
-f rawvideo -pix_fmt %(pix_fmt) -s %(width)x%(height) -r %(fpsnum)/%(fpsden) -i - -c:v ffv1 %(tempvideofile)

prores
-f rawvideo -pix_fmt %(pix_fmt) -s %(width)x%(height) -r %(fpsnum)/%(fpsden) -i - -c:v prores_ks -profile:v 4 -pix_fmt yuv444p10 "%(tempvideofile)"
Does newer ffmpeg versions encode prores hq with the correct matrix metadata or is it always 470bg ?

[anton_foy]

@poisondeathray
You solved it!!! Thanks again, now I can make my movie :)
FFMPEG now reports it correctly as yuv444p10le.
Enormous difference colour grading the 10 bit version compared to the 8 bit one.

[Cary Knoop]

@poisondeathray
You solved it!!! Thanks again, now I can make my movie :)
FFMPEG now reports it correctly as yuv444p10le.
Enormous difference colour grading the 10 bit version compared to the 8 bit one.
If the source is 8-bit it is not going to make any difference.
The same with converting 4:2:0 to 4:4:4, while there are some specific reasons this might be a good idea it usually is not.

[anton_foy]

@Cary Knoop
Filtering and denoising 8 bit material in 16 bit makes a great difference.

[Cary Knoop]

@Cary Knoop
Filtering and denoising 8 bit material in 16 bit makes a great difference.
I would not use the word great, the advantage is minimum.

I actually recommend using 32-bit floats as much as possible for internal processing. Upsampling to 4:4:4 right before you encode is totally meaningless if you want to process the results into Resolve.

Looking at your workflow I am not quite sure the result will actually be an improvement. Especially the "shine" process raises some questions with me. But I await the pre and post comparison.

Also this caught my attention:
Stream #0:0: Video: rawvideo (Y3[0][10] / 0xA003359), yuv444p10le, 640x480, 24 fps, 24 tbr, 24 tbn, 24 tbc

640x480 for a 4k source? Quite a waste if you ask me.

If that is your target resolution, then there is a far better way to turn your 4:2:0 into 4:4:4 by resizing the the luma and chroma planes differently.

[anton_foy]

@Cary Knoop
Do it your way thats fine and yes I get a huge improvement. Now with the 10 bit Prores I get I can finally use the 3DLuts on it without introducing horrible banding and creeping noise I got with the old Avisynth with maximum 8 bit support.
The "shine" is like a glow, instead of just blurring the image loosing detail I overlay an expanded blur to shine over like a physical "pro mist" filter kind of. I still keep the detail. Just watch the clips if you are interested ;) https://we.tl/t-yOeb6Tolr4

[poisondeathray]

640x480 for a 4k source?


640x480 was just the blankclip() default resolution for the debugging test script - to see if there was an issue with this ffmpeg binary, or avisynth environment

[Cary Knoop]

@Cary Knoop
Do it your way thats fine and yes I get a huge improvement. Now with the 10 bit Prores I get I can finally use the 3DLuts on it without introducing horrible banding and creeping noise I got with the old Avisynth with maximum 8 bit support.
The "shine" is like a glow, instead of just blurring the image loosing detail I overlay an expanded blur to shine over like a physical "pro mist" filter kind of. I still keep the detail. Just watch the clips if you are interested ;) https://we.tl/t-yOeb6Tolr4
The "non-processed" video is virtually pitch black and is 444, clearly this is not the straight out of camera clip.

Just take a second from the out of camera clip by for instance doing:

ffmpeg -i clip.mov -ss 10 -t 1 -c:v copy test.mov

This will take one second from the clip at the 10 second marker. Leaving -ss out will take the 1 second from the start.

[anton_foy]

@Cary Knoop
The "non-processed" video is virtually pitch black and is 444, clearly this is not the straight out of camera clip.

Just take a second from the out of camera clip by for instance doing:

ffmpeg -i clip.mov -ss 10 -t 1 -c:v copy test.mov

This will take one second from the clip at the 10 second marker. Leaving -ss out will take the 1 second from the start.
Ah crap, I see that now. I was running it through avisynth without any filters or any conversion but something must have happened (me tired probably).

Thanks, I will use that (ffmpeg -i clip.mov -ss 10 -t 1 -c:v copy test.mov) tomorrow when I get home and post a link.

[kolak]

Does newer ffmpeg versions encode prores hq with the correct matrix metadata or is it always 470bg ?

You can control this in both places: ProRes private headers and MOV headers (if MOV used).
ProRes decoder/encoder got some updates recently and they are bit better now, though still not perfect when it comes to details (compared to Apple reference encoder).

[age]

If the source is 8-bit it is not going to make any difference.
The same with converting 4:2:0 to 4:4:4, while there are some specific reasons this might be a good idea it usually is not.
Definitely wrong.
It's better to use high bitdepth for levels limited to full conversion, chroma upscaling, rgb conversion and of course filtering.

[Cary Knoop]

Definitely wrong.
It's better to use high bitdepth for levels limited to full conversion, chroma upscaling, rgb conversion and of course filtering.
Yes, there are advantages of doing internal processing with a higher bit depths. I personally use float32 for internal processing.

But an 8-bit source is not going to be magically 10 bits or higher due to processing. And a 4:2:0 source is not going to be magically 4:4:4.