Emulating MadVR's HDR to SDR tonemapping using FFMPEG

[Atak_Snajpera]

sample
http://www.4ktv.de/sony-camp-2016-hdr-version-download/

MadVR - frame 3596
http://i.cubeupload.com/uwgYPy.png

FFMPEG
ffmpeg.exe -ss 00:01:00.12 -i Sony_4K_HDR_Camp.mp4 -frames 1 -vf zscale=transfer=linear,tonemap=clip,zscale=transfer=bt709,format=yuv420p c:\users\dave\desktop\ffmpeg.png

http://i.cubeupload.com/SLFmvU.png

Does anybody know how to tweak default values (param). I really do not know where those values should be put in command line.
https://ffmpeg.org/ffmpeg-filters.html#tonemap

[richardpl]

It is rocket science:

ffmpeg.exe -ss 00:01:00.12 -i Sony_4K_HDR_Camp.mp4 -frames 1 -vf zscale=transfer=linear,tonemap=tonemap=clip:param=1.0:desat=2:peak=0,zscale=transfer=bt709,format=yuv420p c:\users\dave\desktop\ffmpeg.png

[Atak_Snajpera]

Thank you! :)

[Atak_Snajpera]

I have another question. Why I'm getting this error when I try to work from .avs script.

SCRIPT
LoadPlugin("C:\Users\Dave\Documents\Delphi_Projects\RipBot264\_Compiled\Tools\AviSynth plugins\ffms\ffms_latest\x64\ffms2.dll")
video=FFVideoSource("E:\_Video_Samples\mp4\Sony_4K_HDR_Camp.mp4",cachefile = "C:\Temp\RipBot264temp\job1\Sony_4K_HDR_Camp.mp4.ffindex")
return video

"ffmpeg.exe" -i "C:\Temp\RipBot264temp\job1\job1.avs" -vf zscale=transfer=linear,tonemap=tonemap=hable:param=1.0:desat=0:peak=10,zscale=transfer=bt709,format=yuv420p C:\Temp\test.y4m

http://i.cubeupload.com/VKXwga.png

When I use file directly instead of script it works fine
"ffmpeg.exe" -i "E:\_Video_Samples\mp4\Sony_4K_HDR_Camp.mp4" -vf zscale=transfer=linear,tonemap=tonemap=hable:param=1.0:desat=0:peak=10,zscale=transfer=bt709,format=yuv420p C:\Temp\test.y4m

http://i.cubeupload.com/u4ecVb.png

[dipje]

out of curiousity? did you try the other tonemapping algorithms? From the description 'mobius' seems promising.

About your avs filter not seeing any path.. I'm _a bit_ stumped.. yuv420p10le in both cases.

What I'm guessing here (really guessing):
The tonemap filter only accepts float values, so it's trying to go from yuv420p10le to some yuv444-floating point.
I'm guessing (since yuv444p32f or something is not in my pix_fmts list) that the only filter capable of producing float-video is the zscale filter.
And the zscale filter refuses to work if you don't say what kind of color-matrices and such it needs for yuv / colorspace conversion.

In your original .mp4, that information is available from the stream (you see it listed after the yuv444p10le) but the .avs reader in ffmpeg is not reporting any colorspace/color-primaries information zo scale refuses to work.

[Atak_Snajpera]

I tried this ...
ffmpeg.exe -i C:\Temp\RipBot264temp\job1\job1.avs -color_range 1 -color_primaries bt2020 -color_trc smpte2084 -colorspace bt2020nc -vf zscale=transfer=linear,tonemap=tonemap=hable:param=1.0:desat=0:peak=10,zscale=transfer=bt709,format=yuv420p -strict -1 C:\Temp\test.y4m
... and still I'm getting the same error.

http://i.cubeupload.com/YaB3cb.png

[nevcairiel]

The error is from z.img, not FFmpeg, so its likely some missing information somewhere.
You can probably override the input information by providing the -color* options before the -i option. Anything after -i refers to the output.

[dipje]

nah that won't work. -color_range parameters etc are for container work or something in ffmpeg, the filters never see them. You have to tell it the zscale filter itself (remember, I'm still guessing). zscale has rangein/primariesin/matrixin/transferin parameters

BTW, recompiled my ffmpeg to get tonemap, and did a few tests with mobius/hable and so far they're not yet the answer I was hoping for :(.

[nevcairiel]

I would generally recommend to avoid AviSynth because of all these issues however. Even more so with HDR, all the extra HDR metadata is also lost from AviSynth, and I don't think there even is any way to specify it manually (although I don't know if zscale uses that, but if not, it may in the future)
If FFmpeg is allowed to see the input file directly, then it'll all fall in place magically.

[dipje]

-vf zscale=tin=smpte2084:min=bt2020nc:pin=bt2020:rin=tv:t=smpte2084:m=bt2020nc:p=bt2020:r=tv,zscale=t=linear,tonemap=tonemap=clip,zscale=t=bt709,format=yuv420p

this works! (man that took some time to figure out :S).

With the first 'zscale' I'm basically doing a sort of no-op, but I'm telling zscale what kind of input data it is, and I'm telling it to convert it to 'the same'. But now, the information of what it is is known in the ffmpeg filter chain, so the next zscale=t=linear will now work (and convert it to gbrpf32le apparently).

if I do it in a single zscale filter instance, I'm hitting issues that the zscale implementation inside of ffmpeg has no way to convert to RGB.. and the linear information can only be RGB (a 'matrix=rgb' is missing but needed for this apparently).

I've tested this on a .y4m file that (just like the Avisynth reader) has no information on what kind of format the yuv stream is in besides the planar configuration.

edit: Actually, mobius and hable tonemapping options (even reinhard) don't look that bad now. With all 3 of them you clearly see a bit more detail / local contrast in the water in the opening at that 1 minute opening mark for example. Still have the feeling it's a bit too desaturated

[dipje]

Think I got the desaturation a bit figured out. You / we never change the color primaries and/or the colormatrix. That means the primaries are still in bt2020. We tonemap the brightness but not much else.

There might be a simpler method, but what I do:
zscale=t=linear,format=gbrpf32le,zscale=p=bt709,tonemap=tonemap=hable,zscale=t=bt709:m=bt709:r=tv

(after the whole 'setting the correct input format' thing).

The first zscale=t=linear turns the YUV format into linear-space-RGB. (Linear space is only a thing in RGB). But the colorprimaries are still in bt2020. (Note, not the colormatrix. The colormatrix is a YUV only thing. But the colorprimarie is basically your colorspace).
Then comes the format=gbrpf32le to turn it all into floating-point RGB. _Then_ do we tell zscale to change the primaries to bt709. According to the zimg documentation changing primaries need to be done in linear space anyway, and this way we make sure the data is linear first. Because we're changing from bt2020 to bt709, we actually start to clip the data at very saturated colors. This is why we first convert to floating point, so this data isn't lost, it's just out-of-gamut.
Then we do the tonemapping, and because it now has data that falls outside of the 0.0 - 1.0 range, it can do it's thing and start desaturating the colors where it thinks it's needed.

After the tonemapping, we use zscale to go back to 'the usual' SDR config of bt709 gamma transfer, and switch to YUV with bt709 colormatrix (the primaries were already in bt709) and we make sure the go to limited range YUV. You may also set the dither options for zscale because we're now requesting it to go down to 8bit. Zscale gets the data from the filter-chain that 'yuv420p' is requested after it and it does the conversion (I believe this is how it works, thus not using the ffmpeg scaler / swscale for this thing). IF you put a 'showinfo' filter between the last zscale and the final 'format=yuv420p' you'll see that the data is already yuv420p after zscale (so I'm guessing zscale is doing the conversion).

edit:
in the 'zscale=t=linear' filter, you might want to add the npl parameter. So it becomes something like 'zscale=t=linear:npl=100'.
I have the feeling it maps the smpte2084 curve and peak data to something that is more alike for a 100 nits screen. This will cause a lot of values to go out of scope (float < 0.0 or float > 1.0, specially the last thing). But that is where the tonemapping filter comes in: How to deal and what to do with values that are outside the display range.
The 'peak=' parameter to the tonemapping filter doesn't seem to have much effect on my side (at least when using hable algorithm), while with 'npl' I can control the final output brightness. The higher the npl= value, the darker the final output video (Because it's meant for displays that have / can do a higher brightness ).

If I use 'hable' and NOT set any peak information and read directly from the Sony .mp4 file, it goes bonkers. I need to set npl=100000 in the first zscale step to get a normal looking video there.

If I use a .y4m as input and don't set the npl, it looks OK, so there are quite some differences in how things are handled.. the bt2020nc is for 'non constant luminace' right? Is there extra metadata in the stream that constantly tells the maximum luminance for a scene??

[Atak_Snajpera]

dipje you are my hero! Thank you very much for your help! I would never figure that on my own.

I'm posting example command line for those who what to convert HDR to SDR and pipe result to encoder.

ffmpeg.exe -loglevel panic -i script.avs -vf zscale=tin=smpte2084:min=bt2020nc:pin=bt2020:rin=tv:t=smpte2084:m=bt2020nc:p=bt2020:r=tv,zscale=t=linear:npl=100,format=gbrpf32le,zscale=p=bt709,tonemap=tonemap=hable:desat=0,zscale=t=bt709:m=bt709:r=tv,format=yuv420p -strict -1 -f yuv4mpegpipe - | x264_x64.exe --stdin y4m --output "C:\video.264" -

[dipje]

all HDR movies are not made the same, so you'll still have to check movie by movie, or even scene by scene how it turns out. There are scenes in the sony demo for instance which I think are too dark.. or I'm tempted to up the 'shadows' slider in Lightroom so to speak :).
But other scenes are clearly very bright and the algorithm is working to map it, so don't really know what to do about that.

[Atak_Snajpera]

It looks like desat=0 works the best. No idea why by default desat is enabled.
The default of 2.0 is somewhat conservative and will mostly just apply to skies or directly sunlit surfaces. A setting of 0.0 disables this option.

desat = 2 (default)
http://i.cubeupload.com/qyqDEU.png
desat = 0 (disabled)
http://i.cubeupload.com/uyHgif.png

Another reason why desat should be disabled!

desat = 2 (default)
http://i.cubeupload.com/4m2lD4.png

desat = 0 (disabled)
http://i.cubeupload.com/y1g1FR.png

[kolak]

So you say that this crazy oversaturated/unreal footage of guy next to fire is better?

[Atak_Snajpera]

Yes, sir!

[dipje]

like the docs say, desat=2 is taking it easy. I would turn it up higher to desaturate places where SDR can't cope with the color gamut.... I would _not_ turn it off... but to each his/her own I guess.

The purpose of desat is too _lower_ saturation in places where otherwise the saturation would be so high that you won't see any details. I would rather see 'less red' in the coat, but actually see the details and movement in the coat. Not just a 'saturated blob of red' which is undefined... just very red.

If you like desat=0, go ahead and use it... but please use it in your own personal encodes... do not bake it into a GUI or tool as default or something :S.

[Atak_Snajpera]

desat destroys image. Desat must be disabled. Period!
http://i.cubeupload.com/34c2yw.png

[shekh]

desat destroys image. Desat must be disabled. Period!


This is obviously not the intended effect. Maybe a bug in zscale or something else?

[dipje]

was the effect I saw with hable and mobius.. I would try tweaking other parameters to fix glitches like that but keep desat at >= 2. But, once again, your choice.

[Selur]

@Atak_Snajpera: Thanks for the example!
----
@all: Is there a way to do this in Vapoursynth? Only thing I saw was the HDR10 to SDR with Hable tone-mapping (https://forum.doom9.org/showthread.php?t=174415)'-thread.
I also read somewhere that tone-mapping basically is wrapping zimg, so this should be possible with Vapoursynth (for someone who knows their way around Python&Vapoursynth), but is there a script out there which basically brings the tone-mapping functionality to Vapoursynth?

Cu Selur

[dipje]

zimg is the scaling library built into vapoursynth. So the vs.resize.xxxx functions are Z.Img routines. (And I used a lot of Vapoursynth's Resize docs in figuring out stuff for ffmpeg zscale filter :P).

So after using zimg / vs.resize to convert to linear RGB in floating point, you can tonemap with basic math in Python if you know the algorithm. The thread about it you're already active in: https://forum.doom9.org/showthread.php?t=174415

[Kef71]

ITU has released recommendation for HDR -> SDR conversion. You can find it here: https://www.itu.int/pub/R-REP-BT.2390-3-2017

Look at page 19:

5.4.1 Mapping to display with limited brightness range

[dipje]

Do note that they're talking about taking the full reference signal (between basically 0 and 10.000 nits) and display it on a normal HDR screen (which for instance can take a maximum of 1.000 nits).

That being said, in figure 20 they show multiple lines for target displays with different brightness capabilities, and they show a blue line for a screen with 100 nits max, which is a nice SDR target.

I'm no math expert, so not really looking at that, but just the text and the graphs basically describe what looks like a pretty normal 'knee' to map the values? Which I boil down in my mind to 'they are recommending to clip signals lower than a certain threshold, clip signals higher than a certain threshold, and use a s-curve for the left-over signal so it looks nice with rolled-off shadows and rolled-off highlights on the target display'. Am I wrong?

This will not really show any detail that might be present in the signal of 1000 nits or brighter, it just rolls it off and compresses it in the upper region. Algorithms like Hable are ment to recover some of those details, right?

[Kef71]

Do note that they're talking about taking the full reference signal (between basically 0 and 10.000 nits) and display it on a normal HDR screen (which for instance can take a maximum of 1.000 nits).

Agreed, HDR10 supports up to 4,000 nits peak brightness, with a current 1,000 nit peak brightness target, 10-bit color depth and capable of displaying everything in the Rec.2020 color space.

That being said, in figure 20 they show multiple lines for target displays with different brightness capabilities, and they show a blue line for a screen with 100 nits max, which is a nice SDR target.

Yep, agreed. ;)

I'm no math expert, so not really looking at that, but just the text and the graphs basically describe what looks like a pretty normal 'knee' to map the values? Which I boil down in my mind to 'they are recommending to clip signals lower than a certain threshold, clip signals higher than a certain threshold, and use a s-curve for the left-over signal so it looks nice with rolled-off shadows and rolled-off highlights on the target display'. Am I wrong?

AFAIK, you are correct in your assumption.

This will not really show any detail that might be present in the signal of 1000 nits or brighter, it just rolls it off and compresses it in the upper region. Algorithms like Hable are ment to recover some of those details, right?

It is an official recommendation not tied to any specific hardware so yes, there are compromises to be made. Remember that the scale is logarithmic so most of the information maps 1:1 from UHD to SDR. Just need to compress a little bit in the upper and lower regions to get a picture as close as possible to UHD. Hable, Reinhard, Mobius are just slightly different shapes of this s-curve as far as I know.

edit: UHD to SDR blu-ray most likely follow this recommendation.

[Atak_Snajpera]

Single threaded tonemapping is creating huge bottleneck on my 8C/16T CPU. Is there any way to speed up zscale by using multi-threading?
http://i.cubeupload.com/RLMy7N.png

http://i.cubeupload.com/JR9KMb.png

[dipje]

From my Vapoursynth experience it's not zscale that is the speed limiting factor, but I'm only guessing.
ffmpeg isn't very good at multi-threading at all, and I think the tonemap filter is just made as a proof of concept as a single-threaded demo. Guess you will have to ask on the ffmpeg-user list.

[tyee]

dipje you are my hero! Thank you very much for your help! I would never figure that on my own.

I'm posting example command line for those who what to convert HDR to SDR and pipe result to encoder.

ffmpeg.exe -loglevel panic -i script.avs -vf zscale=tin=smpte2084:min=bt2020nc:pin=bt2020:rin=tv:t=smpte2084:m=bt2020nc:p=bt2020:r=tv,zscale=t=linear:npl=100,format=gbrpf32le,zscale=p=bt709,tonemap=tonemap=hable:desat=0,zscale=t=bt709:m=bt709:r=tv,format=yuv420p -strict -1 -f yuv4mpegpipe - | x264_x64.exe --stdin y4m --output "C:\video.264" -

@Atak_Snajpera - I tried your command line and get this error - x264 [error]: could not open input file `-' - Do you know why? I'm using 32 bit ffmpeg and x264 because I have 32 bit avisynth installed.

[Selur]

Remove the '-loglevel panic' to see wether ffmpeg complains about having problems with you avs file,..

[Atak_Snajpera]

See how it is done in ripbot264. Add some HDR footage and see C:\Temp\RipBot264temp\Job1\job1_EncodeVideoPass1.cmd.
Probably you made a typo somewhere...

[tyee]

It's getting further now but still errors -

Input #0, avisynth, from 'Standard.avs':
Duration: 00:01:56.83, start: 0.000000, bitrate: 0 kb/s
Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1280x532, 23.98 fps, 23.98 tbr, 23.98
tbn, 23.98 tbc
[zscale @ 19ce05c0] [Eval @ 0028e6cc] Undefined constant or missing '(' in 'smpte2084'
[zscale @ 19ce05c0] Unable to parse option value "smpte2084"
[zscale @ 19ce05c0] [Eval @ 0028e6cc] Undefined constant or missing '(' in 'bt2020nc'
[zscale @ 19ce05c0] Unable to parse option value "bt2020nc"
[zscale @ 19ce05c0] [Eval @ 0028e6cc] Undefined constant or missing '(' in 'bt2020'
[zscale @ 19ce05c0] Unable to parse option value "bt2020"
[zscale @ 19ce05c0] [Eval @ 0028e6cc] Undefined constant or missing '(' in 'tv'
[zscale @ 19ce05c0] Unable to parse option value "tv"
[zscale @ 19ce05c0] [Eval @ 0028e6cc] Undefined constant or missing '(' in 'smpte2084'
[zscale @ 19ce05c0] Unable to parse option value "smpte2084"
[zscale @ 19ce05c0] [Eval @ 0028e6cc] Undefined constant or missing '(' in 'bt2020nc'
[zscale @ 19ce05c0] Unable to parse option value "bt2020nc"
[zscale @ 19ce05c0] [Eval @ 0028e6cc] Undefined constant or missing '(' in 'bt2020'
[zscale @ 19ce05c0] Unable to parse option value "bt2020"
[zscale @ 19ce05c0] [Eval @ 0028e6cc] Undefined constant or missing '(' in 'tv'
[zscale @ 19ce05c0] Unable to parse option value "tv"
[zscale @ 19ce05c0] [Eval @ 0028e71c] Undefined constant or missing '(' in 'smpte2084'
[zscale @ 19ce05c0] Unable to parse option value "smpte2084"
[zscale @ 19ce05c0] Error setting option tin to value smpte2084.
[Parsed_zscale_0 @ 19ce0b80] Error applying options to the filter.
[AVFilterGraph @ 031b18e0] Error initializing filter 'zscale' with args 'tin=smpte2084:min=bt2020nc:
pin=bt2020:rin=tv:t=smpte2084:m=bt2020nc:p=bt2020:r=tv'
Error opening filters!
x264 [error]: could not open input file `-'

[Atak_Snajpera]

Update your ffmpeg https://ffmpeg.zeranoe.com/builds/

[tyee]

That did it, thanks! This will give me 8 bit output, yes? I thought I read somewhere that ffmpeg did not support 10 bit yet?

Update - confirmed it is 8 bit.

[sneaker_ger]

ffmpeg does support 10 bit input/output. But "format=yuv420p" sets 8 bit. 10 bit equivalent is called "yuv420p10le". (I haven't looked at the complete filter list, this is just the last filter.) And if you pipe to x264cli/x265cli you have to use a 10 bit build and possibly explicity set 10 bit output if it's a multi-depth build.

[SpasV]

dipje you are my hero! Thank you very much for your help! I would never figure that on my own.

I'm posting example command line for those who what to convert HDR to SDR and pipe result to encoder.

ffmpeg.exe -loglevel panic -i script.avs -vf zscale=tin=smpte2084:min=bt2020nc:pin=bt2020:rin=tv:t=smpte2084:m=bt2020nc:p=bt2020:r=tv,zscale=t=linear:npl=100,format=gbrpf32le,zscale=p=bt709,tonemap=tonemap=hable:desat=0,zscale=t=bt709:m=bt709:r=tv,format=yuv420p -strict -1 -f yuv4mpegpipe - | x264_x64.exe --stdin y4m --output "C:\video.264" -

Thank you for the post.
The pipe -f yuv4mpegpipe - | doesn't work, but -f rawvideo - | x265 - does.

The CL generates 3840x2160 rawvideo. If 1920x1080 is needed instead, add one more resizing filter:

format=yuv420p,zscale=s=1920x1080

Thank you for the help with this topic. :)

There is no need for using script.avs to introduce a Source filter. ffmpeg has one.

[sneaker_ger]

The pipe -f yuv4mpegpipe - | doesn't work
It's
-strict -1 -f yuv4mpegpipe - | x265 - --y4m
or
-strict -1 -f yuv4mpegpipe - | x264 - --demuxer y4m
respectively.

(Atak_Snajpera had the -strict -1 part in his example.)

[SpasV]

No, it seems format yuv4mpegpipe is not supported anymore.
Besides, the filter zscale=tin=smpte2084:min=bt2020nc:pin=bt2020:rin=tv:t=smpte2084:m=bt2020nc:p=bt2020:r=tv, added by dipje is not needed with the current FFmpeg version.
It is not only a sort of no-op but it adds additional processing time with any frame. These FFmpeg filters (without this no-op) work at least three times slower than DGTonemap filters.

[sneaker_ger]

No, it seems format yuv4mpegpipe is not supported anymore.
I highly doubt that. Why don't you post any logs?


Besides, the filter zscale=tin=smpte2084:min=bt2020nc:pin=bt2020:rin=tv:t=smpte2084:m=bt2020nc:p=bt2020:r=tv, added by dipje is not needed with the current FFmpeg version.
It is not only a sort of no-op but it adds additional processing time with any frame. These FFmpeg filters (without this no-op) work at least three times slower than DGTonemap filters.
No one said you should use both at the same time. They are alternatives.

[Selur]

No, it seems format yuv4mpegpipe is not supported anymore.
yuv4mpegpipe works fine here with latest builds of ffmpeg&co, as an example:
ffmpeg -y -loglevel fatal -threads 8 -i "F:\TestClips&Co\files\HDR\4K sun HDR test.mp4" -map 0:0 -an -sn -vf zscale=tin=smpte2084:min=bt2020nc:pin=bt2020:rin=tv:t=smpte2084:m=bt2020nc:p=bt2020:r=tv,zscale=t=linear:npl=100,format=gbrpf32le,zscale=p=bt709,tonemap=tonemap=hable:desat=2.000:peak=0.000,zscale=t=bt709:m=bt709:r=tv -pix_fmt yuv420p10le -strict -1 -vsync 0 -f yuv4mpegpipe - | x265 --input - --output-depth 10 --y4m --profile main10 --qg-size 8 --limit-modes --no-open-gop --opt-ref-list-length-pps --radl 2 --lookahead-slices 0 --crf 18.00 --opt-qp-pps --cbqpoffs -2 --crqpoffs -2 --limit-refs 0 --ssim-rd --psy-rd 2.50 --rdoq-level 2 --psy-rdoq 10.00 --aq-mode 0 --deblock=-1:-1 --limit-sao --range limited --colormatrix bt709 --output "E:\Temp\00_35_54_2910_01.265"
works fine here,..

-strict -1
is only needed for 10bit color spaces, not with 8bit 4:2:0
,format=yuv420p -strict -1 -f yuv4mpegpipe

-> I agree remove '-loglevel panic' and post the log if you can't make heads and tail of the error messages of ffmpeg.

Cu Selur

[jhughy2010]

I'm trying to convert an HDR/UHD movie to SDR/FHD. I've stumbled upon this article (https://stevens.li/guides/video/converting-hdr-to-sdr-with-ffmpeg/) and am trying ffmpeg for the first time. I'm a regular user of Handbrake but unfortunately this is my first run with ffmpeg.

I've installed ffmpeg and ran the following command:

ffmpeg.exe -i D:\Videos\Media\Temp\video.mkv -vf zscale=t=linear:npl=100,format=gbrpf32le,
zscale=p=bt709,tonemap=tonemap=hable:desat=0,zscale=t=bt709:m=bt709:r=tv,
format=yuv420p -c:v libx265 -crf 18 -preset slower D:\Videos\Media\Temp\output.mkv

I've renamed the movie file (for simplicity sake) to video. Right now I'm getting the following error: "at least one output file must be specified".

Any help would be greatly appreciated. If someone could give me the exact command to try that would be most helpful. The "video.mkv" file is located in D:\Videos\Media\Temp.

Thanks!

[richardpl]

From my Vapoursynth experience it's not zscale that is the speed limiting factor, but I'm only guessing.
ffmpeg isn't very good at multi-threading at all, and I think the tonemap filter is just made as a proof of concept as a single-threaded demo. Guess you will have to ask on the ffmpeg-user list.

Have proof for such harsh claims?

[tebasuna51]

... I'm getting the following error: "at least one output file must be specified"...

Work for me using a unique line like this:
ffmpeg.exe -i D:\Videos\Media\Temp\video.mkv -vf zscale=t=linear:npl=100,format=gbrpf32le,zscale=p=bt709,tonemap=tonemap=hable:desat=0,zscale=t=bt709:m=bt709:r=tv,format=yuv420p -c:v libx265 -crf 18 -preset slower D:\Videos\Media\Temp\output.mkv

Copy/Paste or make a .bat file with a unique line.

[jhughy2010]

Work for me using a unique line like this:


Copy/Paste or make a .bat file with a unique line.

Thank you. I appreciate your help... it appears it's working; however, I'm getting approximately 0.5 fps for the encode. At this rate I figured it would take 96 hours to complete the encode. A few questions: what do I have to do in order to speed up the encode? I'm using an i7-4790k overclocked to 4.6GHz. It is a bit old... I'm willing to upgrade to speed the process up.

Perhaps there is a command that'll speed it up as well? This is my first experience with ffmpeg and/or CLI encoding. Thanks!

[tebasuna51]

I'm getting approximately 0.5 fps for the encode. At this rate I figured it would take 96 hours to complete the encode...

Is a normal speed for encode 4K HEVC with your CPU.
Is more or less the same if you use x265 to encode instead ffmpeg.
You can use other -preset between: ultrafast superfast veryfast faster fast medium slow slower veryslow placebo
to improve the speed obtaining less quality or more size.

Only if you can use some GPU to encode HEVC you can obtain much more speed.

[EmKayVe]

Work for me using a unique line like this:


Copy/Paste or make a .bat file with a unique line.

Is a normal speed for encode 4K HEVC with your CPU.
Is more or less the same if you use x265 to encode instead ffmpeg.
You can use other -preset between: ultrafast superfast veryfast faster fast medium slow slower veryslow placebo
to improve the speed obtaining less quality or more size.

Only if you can use some GPU to encode HEVC you can obtain much more speed.

This worked wonders for me. I changed the preset to "medium" and crf to 20. I was getting around 3 to 4 fps for the encode.

One thing I would like to know with ffmpeg is now to set the audio to encode to AAC 320kbps 5.1 channels. Also, how do you set the target resolution to 1920x1080. I found that in order to make those two adjusments, I am having to utilize handbrake after ffmpeg since I don't know the CLI.

[tebasuna51]

One thing I would like to know with ffmpeg is now to set the audio to encode to AAC 320kbps 5.1 channels. Also, how do you set the target resolution to 1920x1080. I found that in order to make those two adjusments, I am having to utilize handbrake after ffmpeg since I don't know the CLI.

A full recode with ffmpeg is out of scope of this thread (HDR to SDR) and need a answer from a ffmpeg expert (I'm not).

BTW if you want a 1920x1080 SDR output better use a BluRay source.

[EmKayVe]

BTW if you want a 1920x1080 SDR output better use a BluRay source.

Can you be more specific? What do you mean... use a BD rip?

[tebasuna51]

Of course.
A 4K HDR BT-2020 is better than a BD by resolution and colour space, if you reduce resolution to 1920x1080 and colour space to BT-709 you obtain the same quality (or less) than the BD.

BTW to reduce resolution you have the answer in previous post by SpasV:

add at end of video filter: ...,format=yuv420p,zscale=s=1920x1080

[EmKayVe]

Of course.
A 4K HDR BT-2020 is better than a BD by resolution and colour space, if you reduce resolution to 1920x1080 and colour space to BT-709 you obtain the same quality (or less) than the BD.

BTW to reduce resolution you have the answer in previous post by SpasV:

add at end of video filter: ...,format=yuv420p,zscale=s=1920x1080

Ok thank you for your help!

I tried the following and I am getting an error:

ffmpeg.exe -i D:\Videos\Media\Temp\video.mkv -vf zscale=t=linear:npl=100,format=gbrpf32le,zscale=p=bt709,tonemap=tonemap=hable:desat=0,zscale=t=bt709:m=bt709:r=tv,format=yuv420p, zscale=s=1920x1080 -c:v libx265 -crf 20 -preset medium D:\Videos\Media\Temp\output.mkv

The error is "unable to find a suitable output format for zscale=s=1920x1080: Invalid argument.

Any suggestions?

[Monarc]

-vf zscale=t=linear:npl=100,format=gbrpf32le,zscale=p=bt709,tonemap=tonemap=hable:desat=0,zscale=t=bt709:m=bt709:r=tv,format=yuv420p, zscale=s=1920x1080


remove the space before zscale=s=1920x1080

[EmKayVe]

remove the space before zscale=s=1920x1080

Awesome! Thanks that worked.

I'd love to learn more about ffmpeg configuration. Can anyone point me in the right direction for a thread that can help me with this - like audio tracks and such?

So I feel like the HDR to SDR conversion taking place with the configuration in the ffmpeg encode is acceptable. Is there a way to improve upon it or is that pretty much as good as it gets.

[tebasuna51]

https://www.ffmpeg.org/documentation.html

All about command line decoders/encoders/filters...

[EmKayVe]

Ok thank you. I am now only looking for a line to pass through the audio. Then I'll take the resulting file and finish up in Handbrake. I'll just use ffmpeg to tonemap HDR to SDR.

[Beam]

What am i doing wrong?

ffmpeg -ss 00:01:00.12 -i Sony_4K_HDR_Camp.mp4 -frames 1 -vf zscale=t=linear,format=gbrpf32le,zscale=p=bt709,tonemap=tonemap=hable,zscale=t=bt709:m=bt709:r=tv Sony_4K_HDR_Camp_HABLE.png


ffmpeg version 4.1.3 Copyright (c) 2000-2019 the FFmpeg developers
built with FreeBSD clang version 6.0.1 (tags/RELEASE_601/final 335540) (based on LLVM 6.0.1)
configuration: --prefix=/usr/local --mandir=/usr/local/man --datadir=/usr/local/share/ffmpeg --pkgconfigdir=/usr/local/libdata/pkgconfig --enable-shared --enable-pic --enable-gpl --enable-postproc --enable-avfilter --enable-avresample --enable-pthreads --cc=cc --disable-alsa --disable-libopencore-amrnb --disable-libopencore-amrwb --disable-libaom --disable-libass --disable-libbs2b --disable-libcaca --disable-libcdio --disable-libcelt --disable-libcodec2 --disable-libdav1d --disable-libdavs2 --disable-libdc1394 --disable-debug --disable-htmlpages --disable-libdrm --enable-libfdk-aac --disable-libflite --disable-fontconfig --disable-libfreetype --disable-frei0r --disable-libfribidi --disable-gcrypt --disable-libgme --enable-gmp --enable-gnutls --enable-version3 --disable-libgsm --enable-iconv --disable-libilbc --disable-libjack --disable-libklvanc --disable-libkvazaar --disable-ladspa --enable-libmp3lame --disable-liblensfun --enable-libbluray --disable-librsvg --disable-librtmp --disable-libxml2 --disable-lv2 --disable-mbedtls --enable-mmx --disable-libmodplug --disable-libmysofa --enable-nonfree --disable-openal --disable-opencl --disable-libopencv --disable-opengl --disable-libopenh264 --disable-libopenjpeg --disable-libopenmpt --disable-openssl --enable-optimizations --disable-libopus --disable-libpulse --enable-runtime-cpudetect --disable-librubberband --disable-sdl2 --disable-libsmbclient --disable-libsnappy --disable-sndio --disable-libsoxr --disable-libspeex --disable-libsrt --enable-sse --disable-libssh --disable-libtensorflow --disable-libtesseract --disable-libtheora --disable-libtwolame --disable-libv4l2 --disable-indev=v4l2 --disable-outdev=v4l2 --disable-vaapi --disable-vapoursynth --disable-vdpau --disable-libvidstab --disable-libvorbis --disable-libvo-amrwbenc --disable-libvpx --disable-libwavpack --disable-libwebp --enable-libx264 --enable-libx265 --disable-libxavs2 --disable-libxcb --enable-libxvid --disable-outdev=xv --enable-libzimg --disable-libzmq --disable-libzvbi
libavutil 56. 22.100 / 56. 22.100
libavcodec 58. 35.100 / 58. 35.100
libavformat 58. 20.100 / 58. 20.100
libavdevice 58. 5.100 / 58. 5.100
libavfilter 7. 40.101 / 7. 40.101
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 3.100 / 5. 3.100
libswresample 3. 3.100 / 3. 3.100
libpostproc 55. 3.100 / 55. 3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'Sony_4K_HDR_Camp.mp4':
Metadata:
major_brand : isom
minor_version : 1
compatible_brands: isom
creation_time : 2016-02-03T08:01:30.000000Z
Duration: 00:02:07.15, start: 0.000000, bitrate: 75806 kb/s
Stream #0:0(und): Video: hevc (Main 10) (hvc1 / 0x31637668), yuv420p10le(tv, bt2020nc/bt2020/smpte2084), 3840x2160 [SAR 1:1 DAR 16:9], 75620 kb/s, 59.94 fps, 59.94 tbr, 60k tbn, 59.94 tbc (default)
Metadata:
creation_time : 2016-02-03T07:59:49.000000Z
handler_name : Video Media Handler
encoder : HEVC Coding
Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 192 kb/s (default)
Metadata:
creation_time : 2016-02-03T07:59:49.000000Z
handler_name : Sound Media Handler
File 'Sony_4K_HDR_Camp_HABLE.png' already exists. Overwrite ? [y/N] y
Stream mapping:
Stream #0:0 -> #0:0 (hevc (native) -> png (native))
Press [q] to stop, [?] for help
[hevc @ 0x805186600] First slice in a frame missing.
Last message repeated 6 times
[hevc @ 0x8059b6000] First slice in a frame missing.
Last message repeated 6 times
Output #0, image2, to 'Sony_4K_HDR_Camp_HABLE.png':22.77 bitrate=N/A speed=N/A
Metadata:
major_brand : isom
minor_version : 1
compatible_brands: isom
encoder : Lavf58.20.100
Stream #0:0(und): Video: png, rgb48be, 3840x2160 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 59.94 fps, 59.94 tbn, 59.94 tbc (default)
Metadata:
creation_time : 2016-02-03T07:59:49.000000Z
handler_name : Video Media Handler
encoder : Lavc58.35.100 png
frame= 1 fps=0.3 q=-0.0 Lsize=N/A time=00:00:00.01 bitrate=N/A speed=0.00537x
video:808kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown


And this is result: :confused::confused:

https://i.imgur.com/1dptKUt.png

[Gravitator]

What am i doing wrong?
Tried it?

ffmpeg.exe -ss 00:01:00.12 -i Sony_4K_HDR_Camp.mp4 -frames 1 -vf zscale=transfer=linear,tonemap=clip,zscale=transfer=bt709,format=yuv420p c:\users\dave\desktop\ffmpeg.png

[manolito]

What am i doing wrong?

I believe I figured it out... :devil:

Have a look here:
https://forum.doom9.org/showthread.php?p=1898420#post1898420

My HDR to SDR readme for using FFmpeg:
Using FFmpeg to convert HDR sources to SDR:
======================================================================

This video filter call will do it (thanks to dipje and Atak_Snajpera):

zscale=s=704x396,zscale=tin=smpte2084:min=bt2020nc:pin=bt2020:rin=tv:t=smpte2084:m=bt2020nc:p=bt2020:r=tv,zscale=t=linear:npl=100,format=gbrpf32le,zscale=p=bt709,tonemap=tonemap=hable:desat=0,zscale=t=bt709:m=bt709:r=tv,format=yuv420p,colormatrix=bt709:bt601

These parameters are easy to modify for HD output. Edit the "zscale"
parameter for the desired size and remove the "colormatrix" parameter.

This FFmpeg parameter line is meant to be inserted in the "Picture
Settings -> More" field of dmMediaConverter. If you want to use them
in an existing FFmpeg commandline you need to add them as a video
filter and enclose the parameters in double quotes.
Like:
-vf "parameter list"


Note:
If you use the source file directly as the FFmpeg input you may get
borked colors in the output depending on the source format. The safest
way to correct this is to use an AVS script with just the source
filter as the input for FFmpeg (of course only if AviSynth is installed).
Otherwise you have to pre-convert the source to a lossless intermediate
file keeping all the source properties. Then use this lossless file as
the input for FFmpeg and apply the HDR to SDR filter.


Cheers
manolito

[PatchWorKs]

Hi everyone, I honestly don't know if these are overlapping color correction (or equalization ?) problems, but here's another interesting 3ad - I'm partecipating in - about them:

https://forum.videohelp.com/threads/395939-ffmpeg-Color-Range

[damian101]

Think I got the desaturation a bit figured out. You / we never change the color primaries and/or the colormatrix. That means the primaries are still in bt2020. We tonemap the brightness but not much else.

There might be a simpler method, but what I do:
zscale=t=linear,format=gbrpf32le,zscale=p=bt709,tonemap=tonemap=hable,zscale=t=bt709:m=bt709:r=tv

(after the whole 'setting the correct input format' thing).

The first zscale=t=linear turns the YUV format into linear-space-RGB. (Linear space is only a thing in RGB). But the colorprimaries are still in bt2020. (Note, not the colormatrix. The colormatrix is a YUV only thing. But the colorprimarie is basically your colorspace).
Then comes the format=gbrpf32le to turn it all into floating-point RGB. _Then_ do we tell zscale to change the primaries to bt709. According to the zimg documentation changing primaries need to be done in linear space anyway, and this way we make sure the data is linear first. Because we're changing from bt2020 to bt709, we actually start to clip the data at very saturated colors. This is why we first convert to floating point, so this data isn't lost, it's just out-of-gamut.
Then we do the tonemapping, and because it now has data that falls outside of the 0.0 - 1.0 range, it can do it's thing and start desaturating the colors where it thinks it's needed.


Don't do this. Primarily because tonemap desat will clip out-of-gamut pixels to black, which produces ugly artifacts. I also can't quite follow your reasoning, even without desat I can only think of negative possible side effects of this. If you want to prevent desaturation, just set desat to 0. Which is something I don't recommend btw. You always want at least some desaturation for clipped pixels, otherwise some pixels will clip to white way too late, or even never, when one of the three primary colors is completely missing, regardless of brightness This can create unrealistic brightness differences, which is way more distracting than desaturation, especially without reference. Smaller desat value means more desaturation. Don't go below 1 though, at least at 0.2 and lower desat I encountered very visible desaturation on unclipped pixels as well. The default of 2.0 is fine, I normally go lower to catch some edge cases (did some encodes with 1.3 and 1.2, with higher npl lower desat values tend to be a better idea), I rarely mind the desaturation in movies, which might be because I tend to use very high npl (400-700) with mobius, so only extremely bright pixels are desaturated at all. If you want to prevent desaturation at the cost of less accurate brightness differences I recommend a higher desat value, 4 for example. Wouldn't really recommend going higher than that, way too many cases where brighter light sources clip later than less bright light sources because of color differences.

[damian101]

desat destroys image. Desat must be disabled. Period!
http://i.cubeupload.com/34c2yw.png

Well, that's probably because you applied BT.709 primaries before tonemapping.

[damian101]

Single threaded tonemapping is creating huge bottleneck on my 8C/16T CPU. Is there any way to speed up zscale by using multi-threading?
http://i.cubeupload.com/RLMy7N.png

http://i.cubeupload.com/JR9KMb.png

zscale colorspace conversion seems indeed to be the bottleneck. However, it's not much slower than tonemapping in my testing, so not a huge bottleneck.

Some benchmarks:

-vf zscale=t=linear:npl=300,format=gbrpf32le -> 19 fps

-vf zscale=t=linear:npl=300,format=gbrpf32le,zscale=t=bt709:p=bt709:m=bt709 -> 10 fps

-vf zscale=t=linear:npl=300,format=gbrpf32le,tonemap=tonemap=mobius -> 13 fps

-vf zscale=t=linear:npl=300,format=gbrpf32le,tonemap=tonemap=mobius,zscale=t=bt709 -> 9.5 fps

-vf zscale=t=linear:npl=300,format=gbrpf32le,tonemap=tonemap=mobius,zscale=t=bt709:p=bt709:m=bt709 -> 8.3 fps

-vf zscale=t=linear:npl=300,format=gbrpf32le,tonemap=tonemap=mobius,zscale=t=bt709:p=bt709:m=bt709,format=420p -> 7.2 fps

-vf zscale=t=linear:npl=300,format=gbrpf32le,tonemap=tonemap=mobius,zscale=t=bt709:p=bt709:m=bt709:d=ordered,format=yuv420p -> 7.2 fps

-vf zscale=t=linear:npl=300,format=gbrpf32le,tonemap=tonemap=mobius,zscale=t=bt709:p=bt709:m=bt709:d=error_diffusion,format=yuv420p -> 6.3 fps

-vf zscale=t=linear:npl=300,format=gbrpf32le,tonemap=tonemap=mobius,zscale=t=bt709:p=bt709:m=bt709 -pix_fmt yuv420p -> 4.3 fps

-vf zscale=t=linear:npl=300,format=gbrpf32le,zscale,format=yuv420p -> 12 fps

-vf zscale=t=linear:npl=300,format=gbrpf32le -pix_fmt yuv420p -> 6.3 fps

You can also see that swscale format conversion is the slowest operation of them all, so make sure you do it with zscale instead.

mobius and hable are only insignificantly slower than clip

[damian101]

Thank you for the post.
The CL generates 3840x2160 rawvideo. If 1920x1080 is needed instead, add one more resizing filter:

format=yuv420p,zscale=s=1920x1080


Resize before tonemapping to speed tonemapping up a lot.

[Anuskuss]

Is there an alternative for when zscale isn't available?

[huhn]

try libplacebo

[Anuskuss]

try libplacebo

And what if that's not available either (this is not a hypothecial question)?

[huhn]

i don't know a 3rd thing that act as a scaler in ffmpeg.

[manolito]

If you are not sold on FFmpeg and AviSynth+ is an alternative for you, then you can try DGHDRtoSDR by Donald Graft.

Good luck
manolito

[Anuskuss]

Thanks for the tips but neither AviSynth+ nor DGHDRtoSDR are suitable alternatives - I need to use ffmpeg. In the end I decided (https://github.com/intel/media-driver/issues/1833#issuecomment-2227584676) to go for OpenCL:
ffmpeg -i hdr.mp4 -vf format=p010,hwupload,tonemap_opencl=tonemap=mobius:format=nv12:m=bt709:p=bt709:r=tv,hwdownload,format=nv12 thumb-%06d.jpg

The colors are not perfect but better than doing no color conversion at all. If someone knows an ffmpeg filter which improves upon this please let me know.

[GeoffreyA]

The colors are not perfect but better than doing no color conversion at all. If someone knows an ffmpeg filter which improves upon this please let me know.

I gave it a go and managed to get improved results:

ffmpeg -init_hw_device opencl -i HDR.mkv -vf hwupload,tonemap_opencl=tonemap=reinhard:param=0.1:desat=0:format=nv12:t=bt709:p=bt709:m=bt709:r=tv,hwdownload,format=nv12 SDR.mp4

While not perfect, it is not too far from the zscale/tonemap version that I use:

ffmpeg -i HDR.mkv -vf zscale=t=linear:npl=291,format=gbrpf32le,zscale=1920:-1:f=spline36,tonemap=reinhard:param=0.6:desat=0,zscale=t=709:p=709:m=709:r=limited:dither=error_diffusion,format=yuv420p,sidedata=delete OUTPUT

If it's too dark, raise "param" a little at a time. Another problem is that the brightness fluctuates per scene: there is a "threshold" parameter but I didn't test it.

[manolito]

Is there an alternative for when zscale isn't available?

I still do not understand why zscale is not available when you use a plain vanilla current version of FFmpeg. Could you upload a short sample of your source file and specify the desired target size so forum members could do some experimentation?


Cheers
manolito

[Anuskuss]

when you use a plain vanilla current version of FFmpeg

I'm using a fork of ffmpeg called Plex Transcoder which is used by Plex (https://www.plex.tv) (among other things) for thumbnail generation. In it's default state it will create washed out thumbnails for HDR content so I hijacked the command and inserted the OpenCL tonemap filter.

On my desktop I can just do zscale+tonemap which looks way better of course. Maybe someday I'll investigate how the thumbnails are stored on disk and replace them myself, although that sounds tedious and like something I'll eventually forget to do.

[GeoffreyA]

ffmpeg -init_hw_device opencl -i HDR.mkv -vf hwupload,tonemap_opencl=tonemap=reinhard:param=0.1:desat=0:format=nv12:t=bt709:p=bt709:m=bt709:r=tv,hwdownload,format=nv12 OUTPUT


Not perfect but an improvement. If too dark, raise "param" a little at a time.

[Anuskuss]

Not perfect but an improvement. If too dark, raise "param" a little at a time.

While I do appreciate the effort I don't think I'll ever touch the tuning parameters, as I find them too subjective. Sure I can create some beautiful colors with some fine tuning but I have no guarantee that it will look good in the next scene, let alone in a different movie. I'd rather have some "sane" defaults that some color theorist came up with or something that is based on some sort of defined standard. I mean Plex is basically doing the same thing anyway (just with an additional scale_vaapi to get the data into the right format) (https://github.com/intel/media-driver/issues/1833#issuecomment-2226959167) and while I don't think they do the best thing possible, they must've put some thought into it.

[GeoffreyA]

I agree and am of the encoding philosophy to keep the picture "as is." (Except in cases where work is needed, such as debanding anime or fixing problematic material.) What nags me is that, while there are standard ways of tone mapping, for example the ITU-R recommendations, it seems that in practice, one always has to tune the parameters a bit. I wish this were a "solved problem" but we're not there yet and subjectivity is involved.

At present, I have a project where I'm tone mapping the Lord of the Rings 4K remastering down to SDR 1080p, trying to match the remastered 1080p Extended Edition I have access to, and while I'm getting excellent results, it is challenging to match the reference exactly, and getting one scene right may fail another.

[Argaricolm]

I delete my question. To do same as MadVR does we need to do "linear" tonemapping (libplacebo has it).