4K 4:4:4 16-bit CineForm MOV/AVI to 4K 4:4:4 10-bit MP4

[surami]

I think I have to share my experiences with 4K encoding. To show things to the folk on the other end in the best available quality is on the 1st place.

At first I have to start with inspiration (I like timelapse movies), I got it from Tom Lowe (TimeScapes). He revealed to us 2 years ago, that what can we do with a 4K CineForm encoded file in Premier Pro, it's very smooth, search on YouTube.

I started to search few month ago, that how the experts are encoding PP projects directly to x264 (http://www.videolan.org/developers/x264.html) (thanks to Laurent Aimar and the developers; download builds (http://forum.doom9.org/showthread.php?t=89979)) encoded files (it's the best H.264 encoder (http://www.compression.ru/video/codec_comparison/h264_2012/)) and I found a solution. From this site I landed on Advanced FrameServer (http://advancedfs.sourceforge.net) site (thanks to Satish Kumar), so with this solution the PP project can be passed out to a fake AVI in RGB24, RGB32, YUY2 or in UYVY format, the file is around 50kb :). With AviSynth+ (http://avs-plus.net/) (thanks to Ben Rudiak-Gould and the developers, AviSynth (http://avisynth.org.ru/docs/english/faq.htm#q1.2)) you can read in this file and do what you want (convert, filter, etc.). With VirtualDub (http://virtualdub.sourceforge.net/) (thanks to Avery Lee and the developers) + Gradation Curves Plugin (http://members.chello.at/nagiller/vdub/readme.html) (thanks to Alexander Nagiller) you can make screenshot comparisions. Make a similar extreme curve (save it) as seen and described on the CineForm Insider blog (thanks to David Newman). I have to say thanks to Medialooks too, I got to there by watching the files with GraphStudio (http://www.videohelp.com/tools/GraphStudio) (so thanks to Igor János). So after the Avisynth conversion you can render/encode the PP project directly to any video format, what ffmpeg (http://ffmpeg.org/download.html) covers (thanks to Fabric Bellard (http://en.wikipedia.org/wiki/Fabrice_Bellard) and the developers). (I would like to make tests with x265 (http://x265.org/members.html) 444 possibilities too, but it's not ready yet (download builds (http://forum.videohelp.com/threads/357754-%5BHEVC%5D-x265-EXE-mingw-builds))). You can use MeGUI (http://sourceforge.net/projects/megui/) too to read in the AVS file and render (I'm using it to mux the video and audio). Play the final file with with MPC-HC (http://mpc-hc.org/downloads/), and make again screenshot comparisions, here is also recommended to make an extreme curve in PS (save it).

With many tests I got there, that there is a very minimal recognizable visual differenc (the MP4 seems a bit brighter) between the 4K 4:4:4 16bit CineForm MOV/AVI source and the final 4K 4:4:4 10bit MP4 on my 23" Full HD IPS (8bit + 2bit dithered) panel.

!!! Update 2 (correction): the final file is 4K 4:2:0 10bit MP4, because I run into banding issue with 4:4:4.

The final file is 2%, compared to the source. :)

I will update this post with AVS and ffmpeg code.

signpost.avs:
#4K RGB24 (BGR) source from Advanced FrameServer to 4:4:4 or 4:2:0

AVISource("E:\signpost.avi", audio=false).AssumeFPS(25,1)

ConvertToYV24(matrix="Rec709") #4:4:4
#ConvertToYV12(matrix="Rec709") #4:2:0

#resize to 1080p
#Spline36Resize(1920,1080)

Update 1:
It's only an avs4x264mod (https://github.com/astrataro/avs4x264mod) line not ffmpeg, but I will make an update later.

avs_to_mp4.bat:
avs4x264mod.exe --seek-mode safe --x264-binary "E:\x264\x264_10bit.exe" --output "output_4k.mp4" --output-csp i420 --crf 0 --vbv-bufsize 20000 --vbv-maxrate 20000
--colorprim bt709 --transfer bt709 --colormatrix bt709 --input-csp i444 --input-depth 8 "signpost.avs"
pause

[Lenchik]

with this solution the PP project can be passed out to a fake AVI in RGB24, RGB32, YUY2 or in UYVY format
Are you sure you are not loosing 16 bit per channel info of the source outputting to avi in 8 bit per channel formats? If you are loosing you can obtain same quality with freely available cineform decoder and just Virtualdub.

[Motenai Yoda]

Sincerely I still think 4:2:0 is the way to go, specially for 2k and 4k.
Also 4k rgb<->yuv matrix should be bt.2020 not bt.709.

edit: I mean 420 for consumer delivery.

[huhn]

Sincerely I still think 4:2:0 is the way to go, specially for 2k and 4k.
Also 4k rgb<->yuv matrix should be bt.2020 not bt.709.

matrix should be based on the source nothing else.

4k bd are most likely going to be bt. 2020.

if the source was bt 709 the matrix should be too it's that simply.

[kolak]

You can do better than this. Your method is quite simple.
Use vapoursynth with this script:

import vapoursynth as vs
core = vs.get_core()

c = core.avisource.AVISource(r'cinefrom.avi', pixel_type="v210")
c = core.fmtc.resample (clip=c, kernel="blackman", taps=8, css="420")
c = core.fmtc.bitdepth (clip=c, bits=8, dmode=6, ampo=1, ampn=1, staticnoise=0)
c.set_output()

This script will load Cineform file at 10bit (RGB->YUV conversion will happen in Cineform decoder and it's good quality), than it will convert it at very high quality to 4:2:0 and than dither using Floyd-Steinberg method to 8bit with a little bit of added noise. All conversion are done with at least 16bit precision. You can than use vspipe to pass data to ffmpeg or x264/x265 encoder:

vspipe test.vpy - -y4m | ffmpeg - f yuv4mpegpipe -i - -c:v libx264 out.mp4

If you need:

c = core.fmtc.resample (clip=c, kernel="blackman", taps=8, css="420")
c = core.fmtc.bitdepth (clip=c, bits=10, dmode=6, ampo=1, ampn=1, staticnoise=0)

will do 10bit 4:2:0

c = core.fmtc.resample (clip=c, kernel="blackman", taps=8, css="422")
c = core.fmtc.bitdepth (clip=c, bits=8, dmode=6, ampo=1, ampn=1, staticnoise=0)

will do 422 8bit.

ampo and ampn control amount of added noise. 1 gives just a tiny amount which is not visible at all, but helps in the whole bit depths reduction process.
dmode controls dithering method- check fmtconv manual for more info.

[Motenai Yoda]

matrix should be based on the source nothing else.

4k bd are most likely going to be bt. 2020.

if the source was bt 709 the matrix should be too it's that simply.

for 4k standard matrix should be bt 2020, as for hd is bt 709 and for sd bt 601.
so take an sd video (bt 601) upscale it to hd, than play it, colors are all wrong, most players assume bt 709 for hd content and bt 601 for sd.

But I think there's nothing that can do 709 to 2020 for avs actually. (maybe colormatrix should be updated?)


Note: I'm not talking about assign only bt 2020, but convert from 709 to 2020 too.

[kolak]

It does not have to be Rec.2020, as most TVs still expect Rec.709 even for 4K content. Some, like new Samsung 2014 line, can uses Rec.2020, but this has to be manually set in the settings. By default 4K input is converted using Rec.709.
fmtconv can do conversion to Rec.2020.

[foxyshadis]

It's best to just signal it, but x264 doesn't support bt.2020 yet, oddly. It was added to the spec last year. Note that bt.2020 is unspecified for 8-bit, but I suppose you could use it anyway if your screen has the gamut for it and you don't mind the banding.

Edit: Interesting. Reading BT.2020 and BT.2246, I had no idea there were two separate transformations. The old "non-constant luma" is the typical Gamma-scaled RGB->YCbCr, the new "constant luma" method is Linear RGB->Y->Gamma-corrected YBR->YCbCr. Supposedly it gives a more accurate luma, whereas the old method suffers from some darkening when colors are too saturated.

[Reel.Deel]

But I think there's nothing that can do 709 to 2020 for avs actually.

Dither received support for BT.2020 some time ago.



Dither 1.23.0 (http://forum.doom9.org/showthread.php?p=1386559#post1386559):

Partial implementation of the BT.2020 color conversions. Only the non-constant luminance matrix is implemented.

[Lenchik]

It's best to just signal it, but x264 doesn't support bt.2020 yet, oddly.
Komisar build (http://komisar.gin.by/):
--colorprim <string> Specify color primaries ["undef"]
- undef, bt709, bt470m, bt470bg, smpte170m,
smpte240m, film, bt2020
--transfer <string> Specify transfer characteristics ["undef"]
- undef, bt709, bt470m, bt470bg, smpte170m,
smpte240m, linear, log100, log316,
iec61966-2-4, bt1361e, iec61966-2-1,
bt2020-10, bt2020-12
--colormatrix <string> Specify color matrix setting ["???"]
- undef, bt709, fcc, bt470bg, smpte170m,
smpte240m, GBR, YCgCo, bt2020nc, bt2020c

[MasterNobody]

It's best to just signal it, but x264 doesn't support bt.2020 yet, oddly.
It is supported in x264 from rev.2343 (http://git.videolan.org/gitweb.cgi?p=x264.git;a=commitdiff;h=397f60e7f23e2c6ec2cb9b168ebb75cc42983dd7;hp=fa215fc9d77d131595e8b1eda0fc4e9da62c1f94) (from around 30-Jul-2013).

[huhn]

for 4k standard matrix should be bt 2020, as for hd is bt 709 and for sd bt 601.
so take an sd video (bt 601) upscale it to hd, than play it, colors are all wrong, most players assume bt 709 for hd content and bt 601 for sd.

But I think there's nothing that can do 709 to 2020 for avs actually. (maybe colormatrix should be updated?)


Note: I'm not talking about assign only bt 2020, but convert from 709 to 2020 too.

if you play a file in VLC it is treated as bt 601 so you have to convert every file to bt. 601 so VLC shows it right?

i know about the auto detection of renderer base on resolution. but i also know that a at least 2 renderer/player (mpv/madvr)can read the colormatrix.



so we are back at this:

matrix should be based on the source nothing else.

4k bd are most likely going to be bt. 2020.

if the source was bt 709 the matrix should be too it's that simply.

so if you need correct color use a renderer that can use them and put them in the file.

a 709 -> 2020 only lowers PQ

[foxyshadis]

It is supported in x264 from rev.2343 (http://git.videolan.org/gitweb.cgi?p=x264.git;a=commitdiff;h=397f60e7f23e2c6ec2cb9b168ebb75cc42983dd7;hp=fa215fc9d77d131595e8b1eda0fc4e9da62c1f94) (from around 30-Jul-2013).

Oh, dang it, the git mirror I was using is way out of date. No wonder.

I don't know why iec61966-2-4 and iec61966-2-1 were used as primary names instead of xvycc and srgb (or xycc), though.

[surami]

Are you sure you are not loosing 16 bit per channel info of the source outputting to avi in 8 bit per channel formats? If you are loosing you can obtain same quality with freely available cineform decoder and just Virtualdub.

I'm not a video expert and I'm sure, that mathematically (and in theory) I get 8bpc out from the frameserver, but what causes the thing, that there is so minimal differenc (I really made a pixel to pixel comparison) between the source and the final thing? And as I see, the only difference is a minimal brightness. Just make the extreme curve tests for yourself and you will see... I don't know, that how this forced 16bit things are working. Somebody knows about this something? I just started to think in the direction 3D, so maybe here two layers are on eachother with 50% oppacity without left/right shifts?

AviSynth: Layer (http://avisynth.nl/index.php/Layer) function and ChanMix plugin (http://avisynth.nl/index.php/ChanMix).

You can do better than this. Your method is quite simple.
Use vapoursynth with this script:

import vapoursynth as vs
core = vs.get_core()

c = core.avisource.AVISource(r'cinefrom.avi', pixel_type="v210")
c = core.fmtc.resample (clip=c, kernel="blackman", taps=8, css="420")
c = core.fmtc.bitdepth (clip=c, bits=8, dmode=6, ampo=1, ampn=1, staticnoise=0)
c.set_output()

This script will load Cineform file at 10bit (RGB->YUV conversion will happen in Cineform decoder and it's good quality), than it will convert it at very high quality to 4:2:0 and than dither using Floyd-Steinberg method to 8bit with a little bit of added noise. All conversion are done with at least 16bit precision. You can than use vspipe to pass data to ffmpeg or x264/x265 encoder:

vspipe test.vpy - -y4m | ffmpeg - f yuv4mpegpipe -i - -c:v libx264 out.mp4

If you need:

c = core.fmtc.resample (clip=c, kernel="blackman", taps=8, css="420")
c = core.fmtc.bitdepth (clip=c, bits=10, dmode=6, ampo=1, ampn=1, staticnoise=0)

will do 10bit 4:2:0

c = core.fmtc.resample (clip=c, kernel="blackman", taps=8, css="422")
c = core.fmtc.bitdepth (clip=c, bits=8, dmode=6, ampo=1, ampn=1, staticnoise=0)

will do 422 8bit.

ampo and ampn control amount of added noise. 1 gives just a tiny amount which is not visible at all, but helps in the whole bit depths reduction process.
dmode controls dithering method- check fmtconv manual for more info.

Thanks for the info. The source can be RGB24, RGB32, YUY2 and UYVY from the Advanced FrameServer plugin and if I make the conversion in AviSynth as I did with ConvertToYV24 there is minimal brightness issue.

But later I will try the VapourSynth too (if I can figure it out on my computer, till now no succes), I also made some searching in that direction.

Of course the best would be a 16bit display to compare this things...

[kolak]

If your source is 8bit already than you don't need my script. This is for work where you start with high bit depth source for final 8bit delivery.
Premiere frameserver supports only 8bit formats. If you want to use my script you woudl have to export some 10bit file from Premiere, eg v210, ProRes etc. It can be Cineform, but you need to export to Cineform MOV (to preserve 10bit) and than re-wrap to avi for import to vs.

[surami]

Thanks kolak, I understand.

I tested again my singpost.avi (created by the Advanced FrameServer PP plugin), which is in theory 8bit (RGB24). If I set up the input csp in MeGUI mod to BGR, at the log tab I see RGB48LE, so I think I get 16bit out from PP...

The first post is updated now, because I got banding issue if I process 4:4:4 10bit MP4. With the new avs4x264mod line I get a 4:2:0 10bit MP4 with minimal banding, the file size is very small, colors are correct and I can playback with MPC-HC very smoothly.

[surami]

I get 16 bit per channel signpost.avi from Premier Pro (generated by the Advanced FrameServer plugin, I set up RGB24). I just did again an extreme curve comparison in VirtualDub with the help of Gradation Curves plugin between the CineForm 4:4:4 forced 16-bit avi (re-wraped from mov to avi) and the signpost.avi, they are looking the same. Could somebody make a test besides me and check it?

I just wonder if it's correct. The x264 rendering without the AviSynth step (conversion/filtering) would be the best, but how to flag the signpost.avi as a 16bit file and how to convert it with ffmpeg? Most of the softwares are recognizing as an 8bit file, because of the RGB24 setting at the AFS plugin.

[Lenchik]

It is probably still 8 bit per channel. As far as i know, VirtualDub's internal pipeline doesn't work in high bit depth. And images you see are produced by decoder with some conversion.
Maybe this will work as some less destroying conversion. Put in Virtualdub "Video-Color Depth…" menu 4:2:2 YCbCr 10-bit (v210) at both sides of list and run cineform avi. Through File-Export-Use external encode… one can try to feed it to x264.
I am not sure if v210 in such case works only assuming Rec.601 RGB-YUV conversion.

[surami]

I get errors in VirtualDub, so it's not the right way for me. This is my new singpost.avs and 1080p and 4K avs4x264mod.bat:

signpost.avs:
#4K RGB24 (BGR) source from Advanced FrameServer

#AVISource("E:\signpost.avi", audio=false).AssumeFPS(24,1)
AVISource("E:\signpost.avi", audio=false).AssumeFPS(25,1)
#AVISource("E:\signpost.avi", audio=false).AssumeFPS(30,1)

ConvertToYV24(matrix="Rec709")

#resize to 4K for 444 16bit input setting in avs4x264mod
Spline36Resize(7680,2160)

#or resize to 1080p for 444 16bit input setting in avs4x264mod
#Spline36Resize(3840,1080)

4k_avs4x264mod.bat:
avs4x264mod.exe --seek-mode safe --x264-binary "E:\x264\x264_x64_10bit.exe" --output "output_4k.mp4" --output-csp i420 --crf 0 --tune film --vbv-bufsize 15000 --vbv-maxrate 15000
--input-csp i444 --input-depth 16 "signpost.avs"
pause

1080p_avs4x264mod.bat:
avs4x264mod.exe --seek-mode safe --x264-binary "E:\x264\x264_x64_10bit.exe" --output "output_1080p.mp4" --output-csp i420 --crf 0 --tune film --vbv-bufsize 7500 --vbv-maxrate 7500
--input-csp i444 --input-depth 16 "signpost.avs"
pause

[surami]

My newest singpost.avs lines:
#4K RGB24 (BGR) source from Advanced FrameServer

#AVISource("E:\signpost.avi", audio=false).AssumeFPS(24,1)
AVISource("E:\signpost.avi", audio=false).AssumeFPS(25,1)
#AVISource("E:\signpost.avi", audio=false).AssumeFPS(30,1)

# 4:4:4
#ConvertToYV24(matrix="Rec709")

# 4:2:2
#ConvertToYV16(matrix="Rec709")

# 4:2:0
ConvertToYV12(matrix="Rec709")

#resize to 1080p if needed
#Spline36Resize(1920,1080)

+ the ffmpeg.bat-s with the latest ffmpeg build:

prores:
ffmpeg.exe -i "signpost.avs" -c:v prores_ks -qscale:v 1 -pix_fmt yuv422p10 -filter:v unsharp=5:5:0.5:5:5:0.5 "prores_4k_422_10bit.mov"
pause

x264:
ffmpeg.exe -r 25 -i "signpost.avs" -s 3840x2160 -c:v libx264 -b:v 50000k -minrate 50000k -maxrate 50000k -bufsize 50000k -x264-params keyint=25:min-keyint=1 -g 25 -pix_fmt yuv420p
-filter:v unsharp=5:5:0.5:5:5:0.5 -r 25 "x264_4k_420_8bit.mp4"
pause

x265:
ffmpeg.exe -i "signpost.avs" -s 3840x2160 -c:v libx265 -x265-params crf=1:vbv-bufsize=25000:vbv-maxrate=25000 -pix_fmt yuv420p -filter:v unsharp=5:5:0.5:5:5:0.5 "x265_4k_420_8bit.hevc"
pause