Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 23rd July 2018, 10:52   #6221  |  Link
alex1399
Registered User
 
Join Date: Jun 2018
Posts: 50
Quote:
Originally Posted by foxyshadis View Post
If you have a bit-budget, in any way, shape, or form, then you need to turn on VBV. You're only exceeding it because you didn't set up VBV correctly. If you're consistently exceeding it throughout the whole video, you can either raise the profile (longer encoding, bit of a crapshoot on how much it lowers bitrate) or raise the crf (guaranteed bitrate and quality reduction).
OK, I found the culprit of the encoding process. The frame numbers of encoded video reported by ffprobe is one more than the original video material. In 2-pass encoding, the -f null - in the first pass encoding seems to bypass vsync detection and provide a invalid analysis for the coming second pass encoding which has a duplicated frame at the start.

If the -f null - in the first pass encoding is replaced by any dummy output, both output of the first pass encoding and the second pass encoding will have the same duplicated frame and the analysis is valid.

Is it possible to log only these error(warning) into txt file during the batch process, so I could read it when I'm back to screen?
alex1399 is offline   Reply With Quote
Old 23rd July 2018, 11:43   #6222  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 5,849
VSync (Vertical Retrace Synchronisation) is related to a video monitor, a synchronism between monitor and video frame rate ... you probably mean audio/video synchronism. Whatever it means in detail, this will be an issue related to ffmpeg; x265 (as a separate encoder, or as the core library) processes video only and can only rely on the sequence of input frames. If the ffmpeg core serves a duplicate frame, blame ffmpeg.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 23rd July 2018, 12:05   #6223  |  Link
alex1399
Registered User
 
Join Date: Jun 2018
Posts: 50
I'm sorry that confusing, the audio/video sync stuffs that ffmpeg names not the monitor sync stuffs. Does x265 itself have any mechanic to log out error, for example, somebody purposely use a different video at the second pass encoding to do something wrong?
alex1399 is offline   Reply With Quote
Old 23rd July 2018, 13:28   #6224  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 5,849
If x265 uses a statistics file in a 2nd pass which is based on analyzing a different movie in the 1st pass, then it is possible that just the bitrate distribution is not optimal; if you have additional VBV restrictions, it might happen that they fail; if they have a different number of frames, the mismatch may be discovered in the end; but in general, just the quality may vary unexpectedly. There are no security measures to match statistics file and video source (which would be nearly impossible anyway if it comes out of a pipe). Your only workarounds I can imagine right now:

a) let the 1st pass create an output file too;
b) ignore audio during video encoding to avoid this issue if it is not in sync in the source file, always process audio separately, and multiplex it to the final video only

In any case, x265 is not to be blamed.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 23rd July 2018, 14:53   #6225  |  Link
alex1399
Registered User
 
Join Date: Jun 2018
Posts: 50
The second pass encoding just use the slice-type order from the first pass encoding where the first pass intently encodes video A and the second pass intently encodes video B respectively. Both video A and B have the same frame-rate, resolution and amount of frames. the slice-type order is checked by ffprobe -show_frames.

Before I closed the paused batch window during the weekend, it shows something like "[warning]: specified frame type (*) at **** is not compatible with keyframe interval" and "[error]: slice=I but 2pass stats say B", never mind. Now a dummy output is assigned for the first past encoding output.

During debug mode, to record the PSNR statics, options tune PSNR for x265, psnr stats_file for ffmpeg, and move stats_file SomeGenericNameNum are mixed in the batch process. If the result is promising, then the debug stuffs will be canceled and use the default tuning for the rest of the videos.

A bonus demonstration of the mad Adam from RWBY V5E2 for the encoding misplacement
https://i.imgur.com/b6wmHFz.jpg

Last edited by alex1399; 24th July 2018 at 07:47.
alex1399 is offline   Reply With Quote
Old 25th July 2018, 05:01   #6226  |  Link
alex1399
Registered User
 
Join Date: Jun 2018
Posts: 50
No VBV restrictions right now. Just trying to simulate how could those videos badly up-sampled and up-scaled and low-bit-depth to high-bit-depth / high-bit-depth to low-bit-depth conversion back and forth. Now I feel disgusting about those superfluous 10-bit-depth video which was encoded from 8-bit-depth raw source.

A problematic USB stick is fine to reproduce binary corruption / missing in poor network to observe blocking artifacts. --gop-lookahead value larger than --min-keyint value is great on letting x265 produces more noise. Hard-coded artifacts from --dithering and --deblock that make things look not natural : a little subjective. And so on.
alex1399 is offline   Reply With Quote
Old 25th July 2018, 20:56   #6227  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Germany
Posts: 581
Quote:
Originally Posted by alex1399 View Post
Now I feel disgusting about those superfluous 10-bit-depth video which was encoded from 8-bit-depth raw source.
Actually, it makes sense to encode in 10bit starting from an 8bit source. I know that if there's banding already in the 8bit source, there's gonna be banding in the 10bit encode as well, but if I had to deband an 8bit source, I wouldn't encode it back to 8bit unless I really need to. Besides, Main10 is the "de-facto" standard and is widely compatible with players (unlike H.264 High10 that wasn't very supported).
__________________
Broadcast Encoder
LinkedIn
FranceBB is offline   Reply With Quote
Old 26th July 2018, 19:44   #6228  |  Link
MeteorRain
結城有紀
 
Join Date: Dec 2003
Location: NJ; OR; Shanghai
Posts: 528
Quote:
Originally Posted by alex1399 View Post
Now I feel disgusting about those superfluous 10-bit-depth video which was encoded from 8-bit-depth raw source.
Have you ever taken the internal bit-depth / precision loss into consideration? IIRC 8-bit HEVC uses 8-bit internal processing which will effectively reduce the bit-depth to lower than 8-bit. (Correct me if I'm outdated)

Take math as an example. The area of a circle of 4.5, computing in 1-digit after point internal precision is 4.5 * 4.5 -> 20.2, then 20.2 * pi -> 62.6. However computing in 2-digit internal precision gives you 63.58 which is much closer to the real number 63.61725015. Totally different result.

Last edited by MeteorRain; 26th July 2018 at 19:47.
MeteorRain is offline   Reply With Quote
Old 26th July 2018, 22:01   #6229  |  Link
Asmodian
Registered User
 
Join Date: Feb 2002
Location: San Jose, California
Posts: 3,530
That was true in AVC but HEVC uses a high bitdepth for most of the math internally, for all output bitdepths, but it is still a good idea to always encode to 10 bit HEVC.
__________________
madVR options explained
Asmodian is offline   Reply With Quote
Old 27th July 2018, 19:08   #6230  |  Link
MeteorRain
結城有紀
 
Join Date: Dec 2003
Location: NJ; OR; Shanghai
Posts: 528
Quote:
Originally Posted by Asmodian View Post
That was true in AVC but HEVC uses a high bitdepth for most of the math internally, for all output bitdepths
My memory says that, but I did a quick search and couldn't find anything saying x265 is using high bitdepth for internal precision. So not very certain about that part.
MeteorRain is offline   Reply With Quote
Old 27th July 2018, 19:10   #6231  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,542
I also recall reading somewhere that internally everything happens in 16-bit precision.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 28th July 2018, 12:01   #6232  |  Link
Barough
Registered User
 
Barough's Avatar
 
Join Date: Feb 2007
Location: Sweden
Posts: 322
x265 v2.8+56-613074c6714f (GCC 7.3.0, 32 & 64-bit 8/10/12bit Multilib Windows Binaries)

Code:
https://bitbucket.org/multicoreware/x265/commits/branch/default
Barough is offline   Reply With Quote
Old 29th July 2018, 06:11   #6233  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,413
Quote:
Originally Posted by MeteorRain View Post
My memory says that, but I did a quick search and couldn't find anything saying x265 is using high bitdepth for internal precision. So not very certain about that part.
The spec absolutely hates being clear about things mere mortals like to know, but there are a few points: All dct transform processing is from -2^15 to 2^15 (7.4.3.2.2), meaning signed 16-bit. That's about it, though; for purposes of processing they're immediately scaled to bitdepth, and reference prediction & filtering are performed in the bit-depth of the encode. (8.5.3-4 & 8.7.2 has much of the gory details.)
__________________
There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order.
foxyshadis is offline   Reply With Quote
Old 29th July 2018, 15:55   #6234  |  Link
alex1399
Registered User
 
Join Date: Jun 2018
Posts: 50
Thanks for everybody's input. To provide an additional reason why lossless x265 is superior, Video A and B are presented for a trivial scheme : interweave.
First, interweave both video; Second, select and mux the even frame; third, compare it under PSNR measure.

Video A: https://www5.zippyshare.com/v/uVQP4Ccc/file.html
Video B: https://www87.zippyshare.com/v/pkkl1Eph/file.html

x265 crf 0 interweave scheme
ffmpeg -i in.y4m -i out.y4m -filter_complex framepack=frameseq -c:v libx265 -crf 0 output.mp4
ffmpeg -i output.mp4 -vf select=mod(n\,2) -r 24000/1001 -c:v libx265 -crf 0 output1.mp4
ffmpeg -i output1.mp4 -i out.y4m -lavfi psnr=stats_file=psnr.csv -f null -

x265 lossless interweave scheme
ffmpeg -i in.y4m -i out.y4m -filter_complex framepack=frameseq -c:v libx265 lossless=1 output.mp4
ffmpeg -i output.mp4 -vf select=mod(n\,2) -r 24000/1001 -c:v libx265 -x265-params lossless=1 output1.mp4
ffmpeg -i output1.mp4 -i out.y4m -lavfi psnr=stats_file=psnr.csv -f null -

raw lossless interweave scheme
ffmpeg -i in.y4m -i out.y4m -filter_complex framepack=frameseq -f yuv4mpegpipe output.y4m
ffmpeg -i output.y4m -vf select=mod(n\,2) -r 24000/1001 -f yuv4mpegpipe output1.y4m
ffmpeg -i output1.y4m -i out.y4m -lavfi psnr=stats_file=psnr.csv -f null -

A mean-square-error of 0.00 but PSNR of non-inf is quite surprised me in the crf=0 x265 scheme.
Maybe there exists some bit-exact-accuracy options I didn't activate for x265, but crf=0 is definitely not lossless.

Out of topic
x264 crf 0 interweave scheme
ffmpeg -i in.y4m -i out.y4m -filter_complex framepack=frameseq -c:v libx264 -crf 0 output.mp4
ffmpeg -i output.mp4 -vf select=mod(n\,2) -r 24000/1001 -c:v libx264 -crf 0 output1.mp4
ffmpeg -i output1.mp4 -i out.y4m -lavfi psnr=stats_file=psnr.csv -f null -

Last edited by alex1399; 29th July 2018 at 17:18. Reason: x264 re-added for bug tracker
alex1399 is offline   Reply With Quote
Old 29th July 2018, 16:06   #6235  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,542
--crf 0 is not meant to be lossless.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 29th July 2018, 16:29   #6236  |  Link
sneaker_ger
Registered User
 
Join Date: Dec 2002
Posts: 5,384
It is for 8 bit x264. I suspect the error is not in libx264, maybe ffmpeg muxing/demuxing. Either way this has nothing to do with x265. Please move to a new thread or better yet report on ffmpeg bug tracker.

Last edited by sneaker_ger; 29th July 2018 at 16:34.
sneaker_ger is offline   Reply With Quote
Old 29th July 2018, 17:17   #6237  |  Link
alex1399
Registered User
 
Join Date: Jun 2018
Posts: 50
If x265 is fine, I'm fine. Just messing around to seek some ordinary mistakes that common people will make during encoding process. Those rare conditions are not bugs, maybe.
alex1399 is offline   Reply With Quote
Old 30th July 2018, 12:09   #6238  |  Link
K.i.N.G
Registered User
 
Join Date: Aug 2009
Posts: 70
I really like x265 but I seem to be unable to get rid of linear smearing/stretching artifacts when there are fast moving objects in a scene.
Is there a specific parameter targeted at improving this, without increasing the bit rate in other areas (those are fine)?

My settings are:
--crf 17 --preset veryslow --profile main10 --level-idc 5 --output-depth 10 --psy-rdoq 4 --aq-mode 3 --qg-size 64 --qcomp 0.7 --subme 5 --master-display "G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(40000000,50)" --colorprim bt2020 --colormatrix bt2020nc --transfer smpte2084 --max-cll "457,179" --hdr --hdr-opt --deblock -1:-1 --no-sao --no-strong-intra-smoothing

example:

Last edited by K.i.N.G; 30th July 2018 at 12:28.
K.i.N.G is offline   Reply With Quote
Old 30th July 2018, 12:17   #6239  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,542
--qg-size could be something to look at. Lowering it will cause the bitrate to rise though.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 30th July 2018, 12:28   #6240  |  Link
K.i.N.G
Registered User
 
Join Date: Aug 2009
Posts: 70
Quote:
Originally Posted by Boulder View Post
--qg-size could be something to look at. Lowering it will cause the bitrate to rise though.
Thank you, I've lowered it to 16 and will report back when its finished.

I can live with 'slight' bitrate increases (which is perfectly understandable).
K.i.N.G is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 18:39.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.