Best encode settings for almost dark scene

[stormy1777]

So this post, is somewhat similar to prior one:

https://forum.doom9.org/showthread.php?t=175868

where the suggestion to use f3kdb() to deband, that worked great for bright/dark scenes with banding, however, source material is that of barely visible Moon, behind very dark clouds.. Here is a sample image from the video:

http://i.imgur.com/ODzJLFAm.jpg (https://imgur.com/ODzJLFA)

Original clip is about 5 min long from a DSLR (clouds are moving but mostly visible on the moon area, rest appears much dark, although probably not completely dark maybe that is the issue), at 25fps, media info from source is:

Format : MPEG-4
Format profile : Base Media / Version 2
Codec ID : mp42 (mp42/avc1/CAEP)
File size : 421 MiB
Duration : 5 min 2 s
Overall bit rate : 11.7 Mb/s
Performer :
Encoded date : UTC 2018-12-24 17:14:14
Tagged date : UTC 2018-12-24 17:14:14

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4.1
Format settings : CABAC / 2 Ref Frames
Format settings, CABAC : Yes
Format settings, RefFrames : 2 frames
Format settings, GOP : M=3, N=12
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 5 min 2 s
Bit rate : 11.6 Mb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 25.000 FPS
Standard : Component
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.225
Stream size : 420 MiB (100%)
Language : English
Encoded date : UTC 2018-12-24 17:14:14
Tagged date : UTC 2018-12-24 17:14:14
Color range : Full
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709
Codec configuration box : avcC


I was thinking this would be a Piece of CAKE to encode, and that I'd end up with a 1MB file, since most of the area is black, however, with X.264-8bit set to: CRF=18 no change in FPS, and Preset Slower/Tuning=Film, Level=Auto, profile=auto, using VirtualDub2 getting a file size of 587!! much LARGER than original file :)

Media info is:

Format : MPEG-4
Format profile : Base Media
Codec ID : isom (isom/iso2/avc1/mp41)
File size : 587 MiB
Duration : 4 min 26 s
Overall bit rate : 18.5 Mb/s
Writing application : Lavf57.78.100

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L5
Format settings : CABAC / 8 Ref Frames
Format settings, CABAC : Yes
Format settings, RefFrames : 8 frames
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 4 min 26 s
Bit rate : 18.2 Mb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 25.000 FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.351
Stream size : 579 MiB (99%)
Writing library : x264 core 152 r2851M ba24899
Encoding settings : cabac=1 / ref=8 / deblock=1:-1:-1 / analyse=0x3:0x133 / me=umh / subme=9 / psy=1
/ psy_rd=1.00:0.15 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 /
fast_pskip=1 / chroma_qp_offset=-3 / threads=6 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 /
interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=3
/ weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 /
rc_lookahead=60 / rc=crf / mbtree=1 / crf=18.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
Color range : Full
Matrix coefficients : BT.709
Codec configuration box : avcC


it came to 18MB/sec!!! went to CRF=22 filesize 264MB (8MB/sec), and CRF=24 filesize 145 (4MB/sec).


Is it just trial/error and go with what looks still decent? is there any special filter/settings for such a scene?

Thanks in advance.

Curious Stormy.

[kuchikirukia]

It's probably the noise.

https://i.postimg.cc/N0XxsJ3g/ODzJLFA.jpg

You could crush the blacks to get rid of it, or ask in the avisynth forum how to apply a denoiser to only the darkest levels.

[poisondeathray]

Color range : Full


It's a full range clip ; did you adjust the levels properly ?

I would brighten it up and like kuchikirukia says - denoise it

[stormy1777]

Thanks for the tips... I'll need a lot more specifics to make progress... like not sure how to:

- 'crush the blacks' or which VirtualDub's built-in filter does a 'denoise' (the names are not very obvious to non-experts :)
- what is a 'full range clip' (the video is directly from a DSLR, it is shot with colors, and media info data is above.. not sure what that Color: Full means, sorry.. and no, I did not adjust any levels...)

I'll ask on other forums/section as time permits.
thanks for any tips.

Stormy.

[Rocinante]

As what rukia said, and one thing you can do on low bitrate encoding of high noise source is set your psy:rd to 0.0:0.0

[stormy1777]

ok, thanks; posted on the VD forum on how to implement these ideas: https://forum.doom9.org/showthread.php?p=1861095

[FranceBB]

ok, thanks; posted on the VD forum on how to implement these ideas: https://forum.doom9.org/showthread.php?p=1861095

is there a particular reason why you want to do it with the old filters available in VirtualDub?
You could just open an .avs script inside VirtualDub and encode as you generally do.
If you could upload a short sample it would be great so that people can play with it.

Using the single frame you uploaded, it looks like it's correctly flagged as "Full Range" 'cause the black is below 16.

https://i.imgur.com/axvx31Q.png

- what is a 'full range clip'?

Levels can be Limited or Full range (someone call them TV and PC).
TV range is Limited range. PC range is Full range.
Because TVs can't display as many things as PCs, everything that goes on air must be in Limited range, otherwise it will be refused on QC.
Limited range is 16-235 (0.0 - 0.7V), while Full range is 0-255 for 8bit.
The value 0 represents the black in Full Range, the value 16 represents the black in Limited Range.
The value 255 represents the white in Full Range, the value 235 represents the white in Limited Range.
As you can see from Histogram, the waveform of your source exceeds the Limited TV Range and your blacks are below 16. (You can see that it goes to the brown part, which represents values below 16 on the bottom and values above 235 on the top).
Depending on where you are gonna play it (what's your target for your encode) you may wanna convert the levels from Full Range to Limited and then encode it as Limited.


As to the grain, if you are willing to try to use Avisynth as frameserver, you could give MDegrain a shot with something like:


super = MSuper(pel=2, sharp=1)
bv1 = MAnalyse(super, isb = true, delta = 1, overlap=4)
fv1 = MAnalyse(super, isb = false, delta = 1, overlap=4)
bv2 = MAnalyse(super, isb = true, delta = 2, overlap=4)
fv2 = MAnalyse(super, isb = false, delta = 2, overlap=4)
MDegrain2(super,bv1,fv1,bv2,fv2,thSADC=800, thSAD=800)


Another good filter for noise that supports 16bit stacked is dfttest.

As to the banding, I strongly suggest you to work with 16bit precision (16bit stacked) and you can set f3kdb like so:

f3kdb(range=15, Y=45, Cb=30, Cr=30, grainY=0, grainC=0, sample_mode=2, blur_first=true, dynamic_grain=false, opt=3, mt=true, keep_tv_range=true, input_mode=1, input_depth=16, output_depth=8)

Anyway, I can't help you any further without a sample.
If you don't wanna use Avisynth, though, I'll leave it to the Virtual Dub experts in the VD section.

Happy Boxing Day. ^^

[stormy1777]

FranceBB,
Thanks SO much for this amazingly detailed and informative reply. I've spent several hours trying to learn/understand this.. and will return in coming days to check out the mdegrain/f3kdb settings...

I'm not fixated to VD internal filters, just trying the "easy way out", but i guess for higher quality one needs to spend/learn more :)

If I understand the histogram correctly, the two BROWN lines are the edges, the thin white line in the middle is the "middle point", and anything above it means WHITE above 253, and anything BELOW that thin white line means BLACKS are BELOW the 16... I have no clue how to arrive at such a chart, but that is for another day.

Yeah, source is from a DSLR, I doubt this would be "limited", it's not a "TV camera" so to speak.

I cut a 100MB from that clip and uploaded here:

https://mega.nz/#!9MY0BYxJ!HljTnlTeMvRr63IWhGX1kwsmUD7c6tX0JDr-R8QflqY

I'm not sure what/much can be done with it, my primary concern was the filesize grew way out of line with the basic settings.. I've used the "--psy-rd 0.0:0.0" and that helped, but if there are other tricks I'll be glad to try..

Again, thanks for taking the time to explain the difference between full range and limited range, I'm sure others will enjoy this for years to come :)

Stormy.

PS: I'm trying to make videos playable both on TV and PC, but now it seems to be "impossible" :) :) I'm hoping i can "skip" this extra step, and just keep it full range, in hope that one day TVs could show like PCs?? are they not the same thing these days :) :)

[FranceBB]

I'm not sure what/much can be done with it, my primary concern was the filesize grew way out of line with the basic settings.. I've used the "--psy-rd 0.0:0.0" and that helped, but if there are other tricks I'll be glad to try..

The only reason why it has a big file size once you encode with x264 using crf is indeed the noise.
H.264 uses moto-compensation in order to save space, by starting with an Intra (which is the starting frame), then P (partially predicted) or B (totally predicted) blocks (4x4 pixels) / macroblocks (8x8 pixels).
For instance, if you have an object moving across the screen, it would be useless to spend bitrate allocating bits to reproduce it every single frame; instead, the frames of the object are encoded just once and then "moved" across the screen in time.
This makes H.264 save a lot of space by encoding only what's strictly necessary and predicting what's not.
Unfortunately, though, the noise/grain changes frame-by-frame (dynamic) and makes x264 think that the same object is actually a different one, so it doesn't predict it and it has therefore to allocate bits to reproduce it.
By denoising the video, you help the x264 prediction algorithm identify objects and correctly predict them and save space (actually avoid to waste bits), that's why I suggested you to denoise the video first.
There are two denoise methods: spacial and temporal.
Spacial denoise attempts to remove high frequency noise by comparing blocks and macroblocks that are close one to another, while temporal denoise compares the same blocks/macroblocks in time (the frames before and after in time).
It's generally a good habit to do a bit of both, without exaggerate; the reason for that is that spacial denoise might cause wax faces or makes areas too flat, while temporal denoise might cause ghosting (objects moving with a "ghost" following them).
Denoise might also cause banding, that's why it's always good to add a bit of debanding at the end.

In Avisynth I would do:

Using 16bit stacked (Dither Tool)


#Indexing
FFVideoSource("video.mp4")

#From 8bit to 16bit stacked
Dither_convert_8_to_16()

#Denoise with 16bit precision
dfttest(sigma=64, tbsize=1, lsb_in=true, lsb=true, Y=true, U=true, V=true, opt=3, dither=0)

#Debanding with 16bit precision and output 8bit
f3kdb(range=15, Y=45, Cb=30, Cr=30, grainY=0, grainC=0, sample_mode=2, blur_first=true, dynamic_grain=false, opt=3, mt=true, keep_tv_range=true, input_mode=1, input_depth=16, output_depth=8)


Using 16bit interleaved (HDR Core)


#Indexing
FFVideoSource("video.mp4")

#From 8bit to 16bit interleaved
Bitdepth(from=8, to=16)

#Denoise with 16bit precision
hqdn3d(ls=8.0, cs=8.0, lt=6.0, ct=4.5, restart=7)

#Debanding with 16bit precision and output 8bit
f3kdb(range=15, Y=45, Cb=30, Cr=30, grainY=0, grainC=0, sample_mode=2, blur_first=true, dynamic_grain=false, opt=3, mt=true, keep_tv_range=true, input_mode=2, input_depth=16, output_depth=8)


You can play with the settings a bit,but that should do what you need.
After that, you can encode with x264 and it will compress it more easily.

As to me, I'm at work in this very moment, so I have limited time and I can't encode the sample and upload it back, but anyway, the first example (16bit stacked) should do what you need.
Last but not least, the reason why I used 16bit to do denoise and debanding is that denoise is prone to generate banding and in order not to make it worse by filtering at 8bit, I tried to have as much precision as possible and then output at 8bit.
To better understand how to use Avisynth, just look at the wiki as it's very detailed and well-explained.
Anyway, once you install the latest Avisynth version (I suggest you the latest build by pinterf), you just have to download the plugins I used, place them in the plugin folder and just copy-paste what I wrote above in a .txt and rename it .avs.
You can use AVSPmod to take a look at the preview.
ffmpeg, x264 and VirtualDub are all capable of opening .avs files.


PS: I'm trying to make videos playable both on TV and PC, but now it seems to be "impossible" :) :) I'm hoping i can "skip" this extra step, and just keep it full range, in hope that one day TVs could show like PCs?? are they not the same thing these days :) :)

Although Limited Range is still mandatory for broadcasters and official releases alike because it's standard, I gotta say that there's a growing number of TVs that are actually able to reproduce Full Range contents normally. Let's put it this way: it's not mandatory for them to be Full Range complaint so it's up to the manufacturer to support it or not.

[hello_hello]

My 2 cents worth....

I'd only denoise it lightly, otherwise it looks too "washy", and disabling psy doesn't look as good to me, but by resizing to 720p and applying light denoising you can reduce the 108MB sample to 30MB while still using a decent CRF value for x264. It's not as though there's lots of detail, and resizing rather than heavy denoising looks better to me.

For the following script, the levels are converted to limited range. The output file size was initially 39 MB. When I added f3kdb() to the end of the script it dropped to 31 MB, and for GradFun3() it was 35 MB.

--preset slower --tune film --crf 18.0

Moon-Noise-NEW.mkv (http://www.filedropper.com/moon-noise-new) (35 MB)

FFVideoSource("D:\Moon-Noise-CLIP.mp4", cachefile="D:\Moon-Noise-CLIP.mp4.ffindex", threads=1)
ColorYUV(Levels="PC->TV")
ResampleHQ (https://forum.doom9.org/showthread.php?t=160038)(1280,720)
MDegrainNL()

function MDegrainNL(clip c, int "thSAD", int "thSAD2", int "TR", int "BLKSize", int "Overlap", bool "MT", bool "LSB")
{
thSAD = default(thSAD, 150) # Denoising strength
thSAD2 = default(thSAD2, thSAD/2)
TR = default(TR, 1) # Temporal radius
BLKSize = default(BLKSize, 16) # Block size
Overlap = default(Overlap, 4) # Block overlap
MT = default(MT, True) # Internal multithreading
LSB = default(LSB, False) # 16-bit

Super = MSuper(c, MT=MT)
Multi_Vec = MAnalyse(Super, Multi=True, Delta=TR, BLKSize=BLKSize, Overlap=Overlap, MT=MT)
MDegrainN(c, Super, Multi_Vec, TR, thSAD=thSAD, thSAD2=thSAD2, MT=MT, LSB=LSB)
}

[hello_hello]

PS: I'm trying to make videos playable both on TV and PC, but now it seems to be "impossible" :) :) I'm hoping i can "skip" this extra step, and just keep it full range, in hope that one day TVs could show like PCs?? are they not the same thing these days :) :)

For a PC, your media player should automatically expand the limited range levels to full range. HDMI inputs can often be configured for either, especially for computer monitors. DVI and VGA should always be full range. There's probably also a option for your video card. It can be set even if your player expands the levels. They won't be expanded twice (or they shouldn't be).

As long as the output/input levels match, it shouldn't matter if the video is full or limited range (aside from limited range having a little more potential for colour banding), but I'd be converting to limited range. There'd be plenty of hardware players and TVs out there that expect limited range, and even if they can be configured for either, many people wouldn't be aware of it.

The LCD computer monitor I bought a while back defaults to TV levels for the HDMI inputs, even though PCs are traditionally full range. I guess it's because people often connect Bluray players etc these days, and with the HDMI input levels set to full range the picture would look washed out.

For my TV, Samsung labelled the option "HDMI Black Level", with the choices being "normal" and "low". The default of "low" sets the TV's input to expect limited range/TV levels, while "normal" equals full range/PC levels. It's a bit counter-intuitive, but when the TV expects limited levels the picture looks darker than when it's set to full range, so I can see why it's labelled that way (the TV would expand the levels itself when the input is set to limited range, but it doesn't for full range, so regardless of which one would be "correct", limited range/"low" always looks darker than full range/"normal").


http://avisynth.nl/images/Nvidia_control_panel.jpg

[Rocinante]

PS: I'm trying to make videos playable both on TV and PC, but now it seems to be "impossible" :) :) I'm hoping i can "skip" this extra step, and just keep it full range, in hope that one day TVs could show like PCs?? are they not the same thing these days :) :)

I highly doubt that your problem with TV is due to color range, but I seem to notice that your final encode LEVEL is in High L5, which should NOT be supported by most TV.

You can easily fix this by changing your encoding LEVEL to L4.1 or below. And you should get a video that is playable both on TV and PC. Level 4.1 is often considered the highest level you can rely on desktop consumer hardware to support. Blu-ray Discs only support level 4.1, and many non-mobile devices like the Xbox 360 specify level 4.1 as the highest they officially support.


Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L5
Format settings : CABAC / 8 Ref Frames

Command line for setting level is "--level 4.1"

[stormy1777]

Wow, so much information, where to begin.... "newbie" is not enough to describe what I think of my skills in this field.. so much to learn (so little time) ["adopted" from Willy Wonka & Chocolate Factory :)]

FranceBB, thanks for all the details and theoretical text, I'm sure others (certainly I) will refer back to this again... Was not aware of avsPmod, very nice avs-editor/video previewer...

Not sure what the second example was for, as it is a bit scared using things I don't fully (mostly) understand, hence my initial desire to just keep it simple and do it from VD2, that way less likely I can cause a lot of "damage" :)

Anyways, took first example, and it sure did reduce filesize so reduced the denoise to "32": dfttest(sigma=32....

Tried it on full clip, and got some strange numbers... crf-20 (with that avs) filesize 45MB, then crf-15 (with same avs), filesize 26.4MB!?? I was expecting a larger file with lower crf...

Hello Hello Thanks to your valuable input, ended up merging the two together like this:

MDegrainNL(LSB=true)

#Debanding with 16bit precision and output 8bit
f3kdb(range=15, Y=45, Cb=30, Cr=30, grainY=0, grainC=0, sample_mode=2, blur_first=true, dynamic_grain=false, opt=3, mt=true, keep_tv_range=true, input_mode=1, input_depth=16, output_depth=8)


function MDegrainNL(clip c, int "thSAD", int "thSAD2", int "TR", int "BLKSize", int "Overlap", bool "MT", bool "LSB")
{
thSAD = default(thSAD, 50) # Denoising strength
thSAD2 = default(thSAD2, thSAD/2)
TR = default(TR, 1) # Temporal radius
BLKSize = default(BLKSize, 16) # Block size
Overlap = default(Overlap, 4) # Block overlap
MT = default(MT, True) # Internal multithreading
LSB = default(LSB, False) # 16-bit

Super = MSuper(c, MT=MT)
Multi_Vec = MAnalyse(Super, Multi=True, Delta=TR, BLKSize=BLKSize, Overlap=Overlap, MT=MT)
MDegrainN(c, Super, Multi_Vec, TR, thSAD=thSAD, thSAD2=thSAD2, MT=MT, LSB=LSB)
}

entire file came to ~200MB which is reasonable for my needs.

Finally, Rocinante, your tip about the@L5 is very timely, to my amazement, some encodes end up in that "L5"!!

As mentioned, I (still) feel like a newbie :) so just followed the advice in X264 panel, I got "Profile=auto" and "Level=auto", now changing that to "level=4.1", there was a tempting 4.2 in the pull down, but did not go for that bait, left it at 4.1 as the original source video is.

Thanks all, I've moved to next source video :) :)

HNY to all!
Stormy.

[hello_hello]

As mentioned, I (still) feel like a newbie :) so just followed the advice in X264 panel, I got "Profile=auto" and "Level=auto", now changing that to "level=4.1", there was a tempting 4.2 in the pull down, but did not go for that bait, left it at 4.1 as the original source video is.

I assume you're using one of the slower x264 speed presets?
The number of reference frames allowed depends on the video resolution and any specified level, so when you do specify one, x264 limits ref frames as required, even if the speed preset would otherwise use more.

When you don't specify a level, x264 picks one to accommodate the speed preset (number of reference frames), so for 1080p at 24fps, I think it'd give you level 4.1 for the medium preset, level 5 for the slow preset, and for the presets slower than "slow" it'd be level 5.1.

[stormy1777]

Yes, using SLOWER preset...

Now switched to Level=4.1 to avoid producing "non-compliant data", however, looking closely at the source videos, seeing some eare originally: High@L4.1 and some are Baseline@L5. Setting the levels=auto, ended up with all of them at high@L5.. So either I fix it at 4.1, or for each video, find that value, and set it in the codec config dialog, but that's tedious, time consuming and, error prone... so for now, just leaving it to 4.1, hopefully that is good enough :) :)

[stormy1777]

HELPPPP :) :) :)


Ever since I've set Level=4.1 in the x264 config settings (it was auto and produced level 5 encodings) getting this warning on many encodings:

[!] x264 [warning]: MB rate (354960) > level limit (245760)


Maybe it will go away if switch to the 64-bit version of VD2?

Or is this something relating to the LEVEL LIMIT fixed at some value, and the video is too much for that level? in which case it is safe to ignore??

I'm using the 32bit since it was more reliable and allows more plugins, etc. The clips are not massive, just 5-10minutes @25fps HD, not 4K or anything like that :)

Thanks for any tips.

Stormy.

[benwaggoner]

What are your decoder requirements? I doubt that using higher levels, and hence more reference frames, are really going to help that much with noisy content where good references are already hard to find. Using a lower level like 4.0 or 4.1 and setting --vbv-maxrate and --vbv-bufsize to the legal maximum will reduce how high bitrates can peak, often without much loss in quality. Setting --vbv-maxrate is how you can set a maximum bitrate with CRF encodes. Stock x264 doesn't automatically limit values for bitrate and reference frames to their profile @ level maximum, so you'll have to manually set those parameters along with level. Lots of tools like MeGUI and some x264 builds (and x265) will automatically adjust parameters so encodes are legal to the specified profile @ level.

For some relatively easy to test improvements inside of x264, try --aq-mode 3 --nr 500. The built-in quantizer-based denoise in x264 isn't magic, but it's pretty good for this kind of scenario as it denoises what increases bitrate with random noise (it only impacts predicted blocks, not intra blocks, at the quantization stage). Aq-mode 3 gives you a bias towards lower QPs in low luma, which can let you increase your CRF value. The combination of aq-mode 3 and a higher CRF can keep the QP in low luma the same, but allow higher QPs in brighter areas where they aren't needed, saving bits on net.

Encoding in 10-bit can also help with this kind of stuff in H.264, even with 8-bit sources.

[stormy1777]

Thanks benwaggoner.

I'm just an "end-user" trying to create "best possible" videos that will "stand the test of time" and will be usable in 20 years ?? :) or more :)
Doing minor edits with VirtualDub2 adding audio and that's about it, so not sure I have any specific decoder requirement other than I might want to avoid any highly customized "corner cases" like 10bit encoding until it becomes more mainstream.

The error:

[!] x264 [warning]: MB rate (354960) > level limit (245760)

did not show up on the original "dark and noisy" scene, only on later, much brighter sources. It never appeared when LEVEL=AUTO in x264 config panel.

I'm not "fixated" on CRF encode.. Just something that will yield "best possible" outcome in most cases, then on edge conditions I can make manual adjustments based on my (slowly) growing knowledge :)

It sounds like my options are:

1) Level=Auto, and end up with L5 videos, I do not know the full ramifications to that, they seem to play fine on PC's and youtube... but I was advised to stick to level=4.1 to be more compatible.

2) Level=4.1 and figure out the correct values for --vbv-maxrate and --vbv-bufsize for VD2/x264 version, and hard-code that into the config panel, or does that change with each encode??

3) Use a non-CRF encode? and avoid this "limit"? (maybe hit other issues or limits :) ?

4) Use 64bit VD2? will that solve it? (haven't tried yet)

5) Ignore these errors when they appear and hope that nothing too serious happened, so far I've not been able to visually detect any issues with those clips that reported this error.

Maybe #5 and #2 will end up with similar quality video?


FranceBB:

On a different topic, just for reference:

Tried calling f3kdb() in various configs here are some results:

a) f3kdb() crf16: filesize: 342MB.

b) Dither_convert_8_to_16() -> f3kdb(input_depth=16, output_depth=8) crf16: filesize: 342MB (md5 identical to a).

c) Dither_convert_8_to_16() -> f3kdb(input_depth=16, output_depth=8) crf15: filesize: 459MB.

d) f3kdb(range=15, Y=45, Cb=30, Cr=30, grainY=0, grainC=0, sample_mode=2, blur_first=true, dynamic_grain=false, opt=3, mt=true, keep_tv_range=true, input_mode=1, input_depth=16, output_depth=8) crf16: filesize: 472MB.

from that I gather the convert to 16bit is probably useful only for the denoising. Also, the f3kdb with all the rest of the flags produce a larger file, although maybe higher quality, that is hard for ME to know, eyes are "subjective", there is no real measurement to know for "sure", they look the same when played on my monitor :) :)

For time being, if no denoising is needed I just run f3kdb without any arguments..

Thanks for the help!!!

Stormy.

[StvG]

... a) f3kdb() crf16: filesize: 342MB.

b) Dither_convert_8_to_16() -> f3kdb(input_depth=16, output_depth=8) crf16: filesize: 342MB (md5 identical to a).

c) Dither_convert_8_to_16() -> f3kdb(input_depth=16, output_depth=8) crf15: filesize: 459MB.

d) f3kdb(range=15, Y=45, Cb=30, Cr=30, grainY=0, grainC=0, sample_mode=2, blur_first=true, dynamic_grain=false, opt=3, mt=true, keep_tv_range=true, input_mode=1, input_depth=16, output_depth=8) crf16: filesize: 472MB ...

f3kdb() works internally in 16bit. If you don't use another filter in 16bit, no point going to 16bit stacked.

[FranceBB]

On a different topic, just for reference:

Tried calling f3kdb() in various configs here are some results:

a) f3kdb() crf16: filesize: 342MB.

b) Dither_convert_8_to_16() -> f3kdb(input_depth=16, output_depth=8) crf16: filesize: 342MB (md5 identical to a).

c) Dither_convert_8_to_16() -> f3kdb(input_depth=16, output_depth=8) crf15: filesize: 459MB.

d) f3kdb(range=15, Y=45, Cb=30, Cr=30, grainY=0, grainC=0, sample_mode=2, blur_first=true, dynamic_grain=false, opt=3, mt=true, keep_tv_range=true, input_mode=1, input_depth=16, output_depth=8) crf16: filesize: 472MB.

from that I gather the convert to 16bit is probably useful only for the denoising. Also, the f3kdb with all the rest of the flags produce a larger file, although maybe higher quality, that is hard for ME to know, eyes are "subjective", there is no real measurement to know for "sure", they look the same when played on my monitor

For time being, if no denoising is needed I just run f3kdb without any arguments..


Thanks for letting me know.
As long as you are fine with the default settings, there's no problem in using them.

1) Level=Auto, and end up with L5 videos, I do not know the full ramifications to that, they seem to play fine on PC's and youtube... but I was advised to stick to level=4.1 to be more compatible.

Level 5 plays fine on PC assuming you have a good player and the proper hardware acceleration for that level. As to YouTube, it always re-encodes everything you upload (i.e you could even upload something with 150 Mbit/s, it will re-encode it anyway).
The reason why people generally suggest not to go over level 4.1 is that it's considered the "de-facto" standard for many players, as BD disks encoded in H.264 are all encoded with Level 4.1.

2) Level=4.1 and figure out the correct values for --vbv-maxrate and --vbv-bufsize for VD2/x264 version, and hard-code that into the config panel, or does that change with each encode??


Level 4.1 defines a maximum bitrate of 50Mbit/s, so if you wanna set them, you should set your vbv lower than 50000 kbit/s.

4) Use 64bit VD2?

Nope, if I understand correctly, x264 is saying that you are using a certain level but the crf you set would exceed the bitrate available to achieve the target quality, so it's limiting it automatically to the maximum available for your preset.

5) Ignore these errors when they appear and hope that nothing too serious happened, so far I've not been able to visually detect any issues with those clips that reported this error.

Well, in some parts of the video the bitrate is gonna be limited to the maximum available bitrate for the level you selected, so the quality displayed won't be the same (it's not gonna be a "proper constant quality"), but if you didn't notice any difference, then perhaps the difference might be so little that you wouldn't be able to notice it unless you are using metrics like SSIM/PSNR, which I think it's an acceptable result, considering that you are encoding the file for your personal usage, so as long as it looks good to you, you're fine. ^_^

[stormy1777]

Thanks so much friends for reading, and putting out of your time to reply, truly I'm honored, and in other ways I help the world too :)

Loved these answers, will try with --vbv-maxrate 49000 so it will possibly avoid "clipping" in those instants; interestingly, it only happens on few sources, so far 2 out of 15 or so produced this warning/error.

I'll report back once it happens again and the outcome...

[benwaggoner]

Thanks so much friends for reading, and putting out of your time to reply, truly I'm honored, and in other ways I help the world too :)

Loved these answers, will try with --vbv-maxrate 49000 so it will possibly avoid "clipping" in those instants; interestingly, it only happens on few sources, so far 2 out of 15 or so produced this warning/error.

I'll report back once it happens again and the outcome...
Specify --vbv-bufsize as well; just maxrate doesn't do anything about peak bitrate. 49000 is a good value there as well; in HEVC the maximum peak bitrate and VBV size is identical for Main Tier profile@level combinations.

[stormy1777]

Okay, so next time it fails like that run it with:

--vbv-maxrate 49000 --vbv-bufsize 49000

Ok, thanks, will report back when it reproduces again, it seems to be HIGHLY source-dependent....

[benwaggoner]

Okay, so next time it fails like that run it with:

--vbv-maxrate 49000 --vbv-bufsize 49000

Ok, thanks, will report back when it reproduces again, it seems to be HIGHLY source-dependent....
If you care about HW decoder compatibility, you should ALWAYS set --profile, --level, --vbv-maxrate, --vbv-bufsize, and make sure --ref isn't above the max allowed.

I can't think of a good use case for not setting those for lossy encodes.

[stormy1777]

Finally got a source that appear to produce, while still using Level 4.1 for all encodes:

[!] x264 [warning]: MB rate (448800) > level limit (245760)

so, added to VD2 x264 configuration panel:

--vbv-maxrate 49000 --vbv-bufsize 49000

kept everything the SAME, re-encode failed with identical warning and produced a slightly larger file.. 71.9MB vs. 74.3MB.

Confirmed with MediaInfo that the flags are honored (the video component):

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4.1
Format settings : CABAC / 4 Ref Frames
Format settings, CABAC : Yes
Format settings, RefFrames : 4 frames
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 2 min 48 s
Bit rate : 3 378 kb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 55.000 FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.030
Stream size : 67.7 MiB (91%)
Writing library : x264 core 152 r2851M ba24899
Encoding settings : cabac=1 / ref=4 / deblock=1:-1:-1 / analyse=0x3:0x133 / me=umh / subme=9 /
psy=1 / psy_rd=1.00:0.15 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 /
deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-3 / threads=6 / lookahead_threads=1 / sliced_threads=0 /
nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 /
b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 /
scenecut=40 / intra_refresh=0 / rc_lookahead=60 / rc=crf / mbtree=1 / crf=16.0 / qcomp=0.60 / qpmin=0 /
qpmax=69 / qpstep=4 / vbv_maxrate=49000 / vbv_bufsize=49000 / crf_max=0.0 / nal_hrd=none / filler=0 /
ip_ratio=1.40 / aq=1:1.00
Color range : Limited
Matrix coefficients : BT.470 System B/G
Codec configuration box : avcC


Decided to re-encode with the VD2 option to 'convert to fps 25', the error went away!! total filesize grew to 82.6MB (the video portion of media info):

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4.1
Format settings : CABAC / 4 Ref Frames
Format settings, CABAC : Yes
Format settings, RefFrames : 4 frames
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 2 min 48 s
Bit rate : 3 801 kb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 25.000 FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.073
Stream size : 76.2 MiB (92%)
Writing library : x264 core 152 r2851M ba24899
Encoding settings : cabac=1 / ref=4 / deblock=1:-1:-1 / analyse=0x3:0x133 / me=umh / subme=9 / psy=1 / psy_rd=1.00:0.15 /
mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-3 / threads=6 /
lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 /
b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 /
rc_lookahead=60 / rc=crf / mbtree=1 / crf=16.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / vbv_maxrate=49000 /
vbv_bufsize=49000 / crf_max=0.0 / nal_hrd=none / filler=0 / ip_ratio=1.40 / aq=1:1.00
Color range : Limited
Matrix coefficients : BT.470 System B/G
Codec configuration box : avcC

Then dropping the 'vbv*' flags and keeping the convert to 25fps, error did not reproduce, filesize grew a bit to 83MB.

So it seems the limit is not completely bounded by the vbv flags, but the FPS plays a key role too and being too high causes "too much" content for a given time-slice (with certain sources)..

I'm OK with this, the original source is 25 fps, which is run at 55fps to have a fast-motion-effect, and typically down-convert to 25fps, so this is fine!

In summary, for anyone curious, not that it makes a huge difference, these were the findings:


no-vbv fps=55 SIZE: 71.9MB - Limit error
vbv=49K fps=55 SIZE: 74.3MB - Limit error
vbv=49K fps=25 SIZE: 82.6MB - NO limit error
no-vbv fps=25 SIZE: 83.0MB - NO limit error


For time being I'm trying to avoid adding anything extra that is not absolutely required, or that I *fully* understand, so going with the last option, to just down convert fps to 25, lets see if any future source will show up that still breaks the limit bounds :)

Thanks,

Stormy.

[Rocinante]

Quick thought, I think why setting a vbvrate doesn't help in this case, it's because "MB error" is not a bitrate error or peak error, it's "Macroblock per second of decoding speed limit", as for Level 4.1 it is 245,760. Which I would say ignore the error prompt, as it shouldn't affect encoding quality, it's just telling you that x264 will limit the macroblock per second according to level profile limit in decoding speed.

[benwaggoner]

So it seems the limit is not completely bounded by the vbv flags, but the FPS plays a key role too and being too high causes "too much" content for a given time-slice (with certain sources)..
The VBV restrictions are the maximum. Any given encode may not come close to that maximum. A --bitrate 1000 encode probably won't hit level 4.1 VBV limits! Increasing frame rate will increase the odds of hitting the VBV cap, since there is more stuff to encode.

I always set maxrate/bufsize for all encodes. It doesn't hurt ones that wouldn't hit the VBV cap, but it makes the ones that would hit the cap remain compliant to codec spec. A good feature of x265 is that it does this automatically whenever level is specified. There are some x264 builds that do this as well, and I strongly feel that should be the default behavior. Limiting ref frames to what's supported based on Profile, level, and frame size should also be automatic. Encoders just shouldn't produce streams that aren't spec compliant with what the bitstream is flagged as being constrained by.

[stormy1777]

I should probably have started a new thread for this error:

[!] x264 [warning]: MB rate (408000) > level limit (245760)

now got it on a source that was originally taken at 50fps. Despite placing the buffer vbv variables the error still occurs, only way to avoid it is to downsample to 25fps.. is that expected? I don't mind placing vbv variables, however, they don't seem to prevent this pop up at the end of a 30min encode :)

[stormy1777]

one sort of unexpected outcome of 'downsampling' the fps from 50 to 25, is filesize INCREASED?? the 50fps encoded to 317MB, whereas when encoded to 25fps came to 338MB, not a big delta, but still, not very clear why :)

[stormy1777]

On the "flip side", a different 50fps original source encoded to size 243MB, whereas when downconverted to 25fps with rest of settings identical, size *reduced* to 238MB, so I guess this is some sort of unpredictable "noise" related to fps and the actual source material.. Since the only way to avoid the error at end of the encode is to downsample to 25fps, will keep doing that for now... The vbv parameters did not prevent the error.

[sneaker_ger]

1. 1080p50 will NEVER fit into H.264 level 4.1. Choose level 4.2 if necessary. The H.264 levels define maximum resolutions/fps.
2. x264 CRF is framerate aware. So with lower framerate it will increase the per-frame bitrate. This can sometimes outweigh the effect of reducing the framerate when the inter-frame compression is high.

[stormy1777]

[QUOTE=sneaker_ger;1864527]1. 1080p50 will NEVER fit into H.264 level 4.1. Choose level 4.2 if necessary. The H.264 levels define maximum resolutions/fps.

Ah, ok, now i see:

http://blog.mediacoderhq.com/h264-profiles-and-levels/

I'll choose 4.2 going forward.. still, the vbv buffer settings did not make the error/warning/pop up go away, only reducing the fps during encoding did the trick... not that it's a big deal for ME, just saying :)

[sneaker_ger]

Vbv only affects bitrate/buffer so naturally it doesn't affect resolution/fps. No vbv setting will ever "fix" this. In the blog you linked there is a table. The first column is "Max macroblocks per second". With 1080p50 you exceed the 245760 allowed for level 4.1.

[stormy1777]

OK, great news, switched to Level 4.2, all else remained same, and error about MB (macroblock) did not occur.. I still downsample in most cases to 25fps, since the high fps is not always needed.. (On very fast movement I'll keep it at 50fps..) Thanks all for the help and tolerance in allowing this thread to go a BIT off topic :) very helpful nonetheless :)

[hello_hello]

The MB warning isn't an error, just a warning to let you know you've exceeded the specified level due to the resolution/frame rate. The alternative would be for x264 to refuse to encode until you specified a higher level, so it doesn't enforce a resolution/frame rate. I think I mentioned earlier that x264 doesn't enforce the VBV settings for the specified level either. Some decoders will cope with higher bitrates, while some don't cope with the official VBV rates for the level they support.

Bluray supports level 4.1, but with further restrictions such as --vbv-bufsize 30000 --vbv-maxrate 40000.

You can leave the level at 4.1 and your player will either play it or not, just as it will or won't if you specify level 4.2 instead :)
Level 4.2 seems to be level 4.1 with a higher decoding speed to support higher frame rates. The maximum bitrate is the same and I think the maximum number of reference frames is the same for both, so setting level 4.2 probably doesn't change the way the video is encoded (don't hold me to that) but it does prevent the warning message.


Level 4.1 defines a maximum bitrate of 50Mbit/s, so if you wanna set them, you should set your vbv lower than 50000 kbit/s.

Actually, it's 50Mbps for the Baseline, Extended and Main profiles. For high profile it's 50Mbps x 1.25, for Hi10P it's x3 and for Hi422P/Hi444PP it's x4. So High Profile 4.1 has these limits.
--vbv-bufsize 78125 --vbv-maxrate 62500

It's mentioned below the table here (https://en.wikipedia.org/wiki/H.264/MPEG-4_AVC#Levels). The table in the blog linked to earlier seems to be wrong in places.

Someone has removed the maximum reference frame values from the Wikipedia table. I wonder why.

It used to display like this:

1,280×720@60 (9)
2,048×1,024@30 (4)

but now looks like this:

1,280×720@60
2,048×1,024@30

[benwaggoner]

Bluray supports level 4.1, but with further restrictions such as --vbv-bufsize 30000 --vbv-maxrate 40000.

Plus --slices, strictly hierarchical pyramid, and others. I strongly recommend using --bluray-compat if you want to work on that hardware. Alas, it turns off Open GOP which would be helpful with 25 frame GOPs. And don't use it if you aren't targeting Blu-ray

[sneaker_ger]

x264 --bluray-compat does not turn off open GOP.

[benwaggoner]

x264 --bluray-compat does not turn off open GOP.
Oh, they fixed it? Awesome.

I wish x264 had the same caliber of user-facing documentation as x265.readthedocs.io.