Seeking suggestion for video encode settings

[iKron]

Last couple of days i have been trying to find a suitable encode settings to encode my BLURAY collection to small hdd.

i used 2 pass vbr. i know some will say why not crf. crf is good but i found in some dark scene video looks really bad. and my goal is to keep it in a decent file size. like 50min video under 2GB.


an one hour video is taking roughly 1hour 50min encode. i really don't have much knowledge about encode. anyway this is the encode settings i used.
https://pastebin.com/TdMMqx1Z

if you have any suggestion to change some encode settings to improve the video quality or reduce the encode time, kindly let me know.

[benwaggoner]

For HD-based playback:
--open-gop will give a small efficiency improvement
--sao --selective-sao 2 is better when trying to get optimal quality at lower file sizes

Setting high tier and no level seems odd and could have compatibility issues on other hardware. With a target ABR of just 4000, I'd do

--profile main10 --level-idc 5.0 --vbv-bufsize 25000 --vbv-maxrate 25000

--aq-mode 3 can help in dark scenes, but likely isn't generally optimal at these low bitrates. I'd try --aq-mode 4

--deblock -1:-1 is a good x264 default, but not for x265 at lower bitrates.

--rskip 2 is generally preferable. It's not clear from the lot if you used it. It's generally simpler to just provide your command line instead.

Given your low target bitrate, I'd throw in --nr-inter 250. That might improve the quality of dark regions some as well.

Ther are tons of other options that will improve quality or allow lower bitrates at the expense of encoding time, of course. My default recommendation is just to start with --preset slower and then tune as needed from there.

[iKron]

thank you so much for giving your time to check the encode settings.

this is one screen comparison: first picture is original and mouse hover is the encoded video
https://screenshotcomparison.com/comparison/23469

i tried vmaf scored which is not much difference slow vs medium and i am not sure if this score is reliable. but time is taking almost 2x more. for 1 hour video it is taking around 1 hour 40min. vbr 2pass encode.

--profile main10 --level-idc 5.0 --vbv-bufsize 25000 --vbv-maxrate 25000
i set maxrate higher because one fighting scene was taking 35~40mbps. what does level-idc 5.0 do? encoded video is: Main 10@L4.1@High


--open-gop will give a small efficiency improvement.
people are saying open-gop can messed up if i don't know exactly what i am doing. so i closed it i am really a noob.

--aq-mode 3 can help in dark scenes, but likely isn't generally optimal at these low bitrates. I'd try --aq-mode 4
i set aq-mode 3 mainly for dark scene. few days ago i didnot know about it. i could not find much information about --aq-mode 4 online

--deblock -1:-1 is a good x264 default, but not for x265 at lower bitrates.
what is your suggestion to use?

--rskip 2 is generally preferable. It's not clear from the lot if you used it. It's generally simpler to just provide your command line instead.
yes i using rskip 2, i think i using it after i read some other thread you mentioned it.


can you please tell me a bit about --nr-inter 250. it is Noise reduction feature right?

meh i wish if there is any site which will explain all popular x265 parameters with example image screenshot, time penalty and output file size.

[rwill]

this is one screen comparison: first picture is original and mouse hover is the encoded video
https://screenshotcomparison.com/comparison/23469


Damn you found the holy grail, the encoded video looks better than the original. Magic.


i tried vmaf scored which is not much difference slow vs medium and i am not sure if this score is reliable.


Forget VMAF, its not a good metric. As soon as there is grain or similar the results become more or less meaningless.




--deblock -1:-1 is a good x264 default, but not for x265 at lower bitrates.
what is your suggestion to use?


--deblock 0:0

[iKron]

original video has lots of grains though...may be this is why encoded one looks better.

[RanmaCanada]

original video has lots of grains though...may be this is why encoded one looks better.

If it has lots of grain, then you're better off to keep it as x264 as HEVC still sucks at grain. It's better than it was, but x264 is still far superior. Maybe once AV1 gets it's collective stuff together and they figure out the grain there, you can move to it.

[excellentswordfight]

If it has lots of grain, then you're better off to keep it as x264 as HEVC still sucks at grain. It's better than it was, but x264 is still far superior. Maybe once AV1 gets it's collective stuff together and they figure out the grain there, you can move to it.
If I understood OP correctly, he is re-encoding original blurays so not sure what you mean with "keep it as x264"? He also seems keen at keeping the re-encodes under 5Mbps, so even if he re-encodes with x264 (which I assume you are sugesting) thats not high enough for x264 to shine when it comes to grain anyway.

And OP hasnt even stated that grain retention is important, and given the size requirement, I think it might not really be. At something like 4Mbps I would definitely give x265 the upper hand.

[RanmaCanada]

If I understood OP correctly, he is re-encoding original blurays so not sure what you mean with "keep it as x264"? He also seems keen at keeping the re-encodes under 5Mbps, so even if he re-encodes with x264 (which I assume you are sugesting) thats not high enough for x264 to shine when it comes to grain anyway.

And OP hasnt even stated that grain retention is important, and given the size requirement, I think it might not really be. At something like 4Mbps I would definitely give x265 the upper hand.

What I mean is, to not bother with them and wait till AV1 gets it together(another year or 2, MAYBE!) as the way it is supposed to handle grain will be far superior (their claim) than x264, and it will allow OP to get the bitrate they apparently demand. If they don't keep the grain, they will just get an unwatchable smoothed out POS, especially at 5mbs.

[benwaggoner]

What I mean is, to not bother with them and wait till AV1 gets it together(another year or 2, MAYBE!) as the way it is supposed to handle grain will be far superior (their claim) than x264, and it will allow OP to get the bitrate they apparently demand. If they don't keep the grain, they will just get an unwatchable smoothed out POS, especially at 5mbs.
AV1 can synthesize grain based on embedded metadata pretty well. This does have a design flaw where grain density is proportional to display or rendition size, so a bigger TV will have finer grain than a phone. Which isn't how grain works physically; it is actual physical particles relative to the source frame. Grain should be proportional to size in the source, and thus should be less visible the smaller the screen. This can be worked around some by doing the grain removal and parameterization uniquely for each resolution, but that adds a lot of compute.

The fundamental challenge is cleanly removing and parameterizing grain from the source, which has been an improving-but-far-from-perfect technology for 25+ years now. And has never been something that's worked well as a fully automated technology without some human tweaking.

How to do optimal grain removal is out of the scope of a codec, and will continue to improve independently of codec standards.

I'm not aware of anyone actually distributing AV1 with grain synthesis at scale yet. While it can work well some of the time, it's not consistent enough to just have as "on" in volume encoding.

[benwaggoner]

If it has lots of grain, then you're better off to keep it as x264 as HEVC still sucks at grain. It's better than it was, but x264 is still far superior. Maybe once AV1 gets it's collective stuff together and they figure out the grain there, you can move to it.
With the right tuning, x265 can do pretty well with grain today. --tune grain itself is ancient and broken and shouldn't be used. But some of the parameters can work well tuned down a bit.

The one real secret trick to encoding grain well is that --rd 6 is often a lot worse than --rd 4 with heavy fine grain, particularly at UHD resolutions. Thus cranking up the preset to slower/veryslow can actually hurt grain quality.

Beyond that, reducing --ipratio and --qpratio, increasing --psy-rd and --psy-rdoq, liberal use of --nr-inter, and --rskip 2 instead of the default can help enormously.