Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 12th May 2018, 23:03   #6101  |  Link
Ma
Registered User
 
Join Date: Feb 2015
Posts: 324
Quote:
Originally Posted by Selur View Post
I'm trying to convert a 8bit pc range clip to 10bit x265 using:
Code:
ffmpeg [...] -f yuv4mpegpipe - | x265 [...]
yuv4mpegpipe is for limited range only. Please use raw video for full range.
Ma is offline   Reply With Quote
Old 13th May 2018, 00:06   #6102  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,814
Quote:
Originally Posted by Selur View Post
I understand some small changes like when using '-vf zscale=rangein=full:range=full' and my lines from above might be a side effect of the video compression, but the compression of the histogram really looks like either a bug in x265 or all the other tools.
I agree . That's why I used crf 8, you'd expect only tiny tiny difference, not that large. It looks like CbCr are affected too

It affects x264 too . vs rawpipe too instead of y4m for fun
Code:
vspipe 1.vpy - | x264_10bit --demuxer raw --input-csp i420 --input-depth 10 --input-res 1280x720 --fps 30000/1001 --crf 8.00 --colormatrix smpte170m --input-range pc --range pc -o vs_raw_x264.264 -
But if it was a toolchain problem, you'd expect lossless to exhibit the same problem

Something is up...
poisondeathray is offline   Reply With Quote
Old 13th May 2018, 02:20   #6103  |  Link
jlpsvk
Registered User
 
Join Date: Dec 2014
Posts: 188
ok. x265 devs. what is the command line to output uhd bd compliant stream?
jlpsvk is offline   Reply With Quote
Old 13th May 2018, 08:44   #6104  |  Link
Jamaika
Registered User
 
Join Date: Jul 2015
Posts: 570
Quote:
Originally Posted by Ma View Post
yuv4mpegpipe is for limited range only. Please use raw video for full range.
Never heard about it. A few threads further users advise against the use of rawvideo.

I test conveters:
Code:
ffmpeg_080518.exe -i "Color Range Test.MOV" -f yuv4mpegpipe -vf scale=1280:720:in_color_matrix=470bg:in_range=full:out_color_matrix=470bg:out_range=full,format=yuv420p10,lutyuv=val:val:val -strict -1 - | 
x264-10bit.exe --demuxer y4m --muxer mp4 --input-csp i420 --input-depth 10 --input-range pc --output-csp i420 --threads 4 --preset veryslow --tune grain --crf 28 --fps 29.970 --keyint 60 --nal-hrd none 
--colormatrix bt470bg --colorprim bt709 --transfer bt709 --range pc 
--output "x264_420p10le_crf28_1.mp4" -
Code:
x264-10bit_2851.exe --muxer mp4 --input-csp i420 --input-depth 8 --input-range pc --output-csp i420 --threads 4 --preset veryslow --tune grain --crf 28 --fps 29.970 --keyint 60 --nal-hrd none 
--colormatrix bt470bg --colorprim bt709 --transfer bt709 --range pc 
--output "x264_420p10le_crf28_2.mp4" "Color Range Test.MOV"
Code:
bpgenc_0.9.8.exe -v -b 10 -f 420 -c ycbcr_bt709 -e x265 -m 9 -q 28 -o x265_420p10le_crf28.bpg rgba.png
Result of the test:
1)bad range,
2)bad range, no latest version 10bit 2901 (2018),
3)good.

Conclusion:
For me, the problem is in BT470bg.

Last edited by Jamaika; 13th May 2018 at 09:28.
Jamaika is offline   Reply With Quote
Old 13th May 2018, 09:33   #6105  |  Link
Ma
Registered User
 
Join Date: Feb 2015
Posts: 324
After some tests it is a bug in ffmpeg (in my opinion).
Sorry for confusions.

Last edited by Ma; 13th May 2018 at 12:23.
Ma is offline   Reply With Quote
Old 13th May 2018, 09:42   #6106  |  Link
Jamaika
Registered User
 
Join Date: Jul 2015
Posts: 570
Quote:
Originally Posted by Ma View Post
If you use -pix_fmt yuv420p or yuv420p10 in ffmpeg it means limited range. Please use yuvj420 or yuv[B]
I know that I use x264, but...
Code:
[yuv4mpegpipe @ 000001d226440300] ERROR: yuv4mpeg can only handle yuv444p, yuv422p, yuv420p, yuv411p and gray8 pixel formats. And using 'strict -1' also yuv444p9, yuv422p9, yuv420p9, yuv444p10, yuv422p10, yuv420p10, yuv444p12, yuv422p12, yuv420p12, yuv444p14, yuv422p14, yuv420p14, yuv444p16, yuv422p16, yuv420p16, gray9, gray10, gray12 and gray16 pixel formats. Use -pix_fmt to select one.
Could not write header for output file #0 (incorrect codec parameters ?): I/O error
Error initializing output stream 0:0 --
or
Code:
[Parsed_format_1 @ 000001587211bec0] Invalid pixel format 'yuvj420p10'
[AVFilterGraph @ 00000158714fd980] Error initializing filter 'format' with args 'yuvj420p10'
Error reinitializing filters!
Failed to inject frame into filter network: Invalid argument
Error while processing the decoded data for stream #0:0
Jamaika is offline   Reply With Quote
Old 13th May 2018, 17:52   #6107  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 5,826
'yuvj420p10'
try:
Code:
-vf zscale=rangein=full:range=full -strict -1 -pix_fmt yuv420p10le
this seems to work for me when using ffmpeg als decoder, but that wouldn't help when using vspipe which poisondeathray tested,..
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 13th May 2018, 18:16   #6108  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,814
Quote:
Originally Posted by Selur View Post
'yuvj420p10'
try:
Code:
-vf zscale=rangein=full:range=full -strict -1 -pix_fmt yuv420p10le
this seems to work for me when using ffmpeg als decoder, but that wouldn't help when using vspipe which poisondeathray tested,..
But you still get problems unless you use lossless, right ?

You can look at histogram classic and there is a big line artifact in the waveform; you wouldn't expect that at low crf values (maybe at high lossy quantizers)

So it's not a pipe or toolchain problem
poisondeathray is offline   Reply With Quote
Old 13th May 2018, 19:04   #6109  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 5,826
Quote:
But you still get problems unless you use lossless, right ?
Not really. (used crf 18 and 8bit)

Uploaded my compare script and the files I use to my GoogleDrive
Argh,.. I used crf 18 and 8bit.
-> will try again with 10bit
You are right there are still 'line artifacts' in the histogram when using 10bit.


Cu Selur
Ps.: As a side note: either I'm making something wrong or x264 is totally broken when handling with pc scale video. (see: https://forum.doom9.org/showthread.php?p=1841998)
__________________
Hybrid here in the forum, homepage

Last edited by Selur; 13th May 2018 at 19:18.
Selur is offline   Reply With Quote
Old 13th May 2018, 19:17   #6110  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,814
In case it's a vpy or algorithm issue (you're using zimg/z.lib in vpy to convert back to 8bit to check at 8bit waveform) , I double checked with the vpy only also with nearest neighbor /Point instead of Bicubic - you don't get those artifacts (when going back/forth in the vpy only) ; you only get those artifacts with x264/x265 lossy encoding (and CRF 8 should be near lossless) . Lossless encoding fixes it, so it's definitely not a pipe or "toolchain" issue .

Last edited by poisondeathray; 13th May 2018 at 19:23.
poisondeathray is offline   Reply With Quote
Old 13th May 2018, 19:38   #6111  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 5,826
Quote:
Lossless encoding fixes it, so it's definitely not a pipe or "toolchain" issue .
Not necessarily, lossless should be YUV4:4:4 and not YUV4:2:0.
Atm. my best guess is that these line artifacts are due to the compression, but it might be a but.
__________________
Hybrid here in the forum, homepage

Last edited by Selur; 13th May 2018 at 19:43.
Selur is offline   Reply With Quote
Old 13th May 2018, 19:44   #6112  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,814
Quote:
Originally Posted by Selur View Post
Not necessarily, lossless should be YUV4:4:4 and not YUV4:2:0.
No; "lossless" is a relative term with respect to whatever your input is

If you have a YUV4:2:0 source, upscaling to YUV4:4:4 is not lossless (unless you use nearest neighbor to up and downscale back to YUV4:2:0)
poisondeathray is offline   Reply With Quote
Old 13th May 2018, 19:45   #6113  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,814
Quote:
Originally Posted by Selur View Post
Atm. my best guess is that these line artifacts are due to the compression, but it might be a but.

That's what it looks like (because lossless encoding fixes it), but it's very bizarre pattern (look at the classic histogram) . Many errors congregate around a low midtone value, creating a line artifact .
poisondeathray is offline   Reply With Quote
Old 13th May 2018, 19:56   #6114  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 5,826
okay, looking at the classic histogram view, I see what you mean, and I guess those might be due to compression artifacts.
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 13th May 2018, 20:02   #6115  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,814
You would expect random quantization artifacts to be more random, not coalesce around 1 value.

And you would expect it to be "less lossy" or higher quality, certainly at CRF 8 .

It might have just been a "perfect storm" conditions . But do you have another clip/scene/section from that camera to test ?
poisondeathray is offline   Reply With Quote
Old 13th May 2018, 20:13   #6116  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 5,826
Quote:
But do you have another clip/scene/section from that camera to test ?
Sadly, no. Got it from a Hybrid user who stumbled over the problem.
(wrote an email to the user whether he could share some additional footage from that source)
__________________
Hybrid here in the forum, homepage

Last edited by Selur; 13th May 2018 at 20:16.
Selur is offline   Reply With Quote
Old 13th May 2018, 22:14   #6117  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,413
Quote:
Originally Posted by Jamaika View Post
Never heard about it. A few threads further users advise against the use of rawvideo.

I test conveters:
Code:
ffmpeg_080518.exe -i "Color Range Test.MOV" -f yuv4mpegpipe -vf scale=1280:720:in_color_matrix=470bg:in_range=full:out_color_matrix=470bg:out_range=full,format=yuv420p10,lutyuv=val:val:val -strict -1 - | 
x264-10bit.exe --demuxer y4m --muxer mp4 --input-csp i420 --input-depth 10 --input-range pc --output-csp i420 --threads 4 --preset veryslow --tune grain --crf 28 --fps 29.970 --keyint 60 --nal-hrd none 
--colormatrix bt470bg --colorprim bt709 --transfer bt709 --range pc 
--output "x264_420p10le_crf28_1.mp4" -
Code:
x264-10bit_2851.exe --muxer mp4 --input-csp i420 --input-depth 8 --input-range pc --output-csp i420 --threads 4 --preset veryslow --tune grain --crf 28 --fps 29.970 --keyint 60 --nal-hrd none 
--colormatrix bt470bg --colorprim bt709 --transfer bt709 --range pc 
--output "x264_420p10le_crf28_2.mp4" "Color Range Test.MOV"
Code:
bpgenc_0.9.8.exe -v -b 10 -f 420 -c ycbcr_bt709 -e x265 -m 9 -q 28 -o x265_420p10le_crf28.bpg rgba.png
Result of the test:
1)bad range,
2)bad range, no latest version 10bit 2901 (2018),
3)good.

Conclusion:
For me, the problem is in BT470bg.
Rematrixing YUV in swscale goes through an RGB intermediate; it's not supposed to rematrix when you pass the same value in and out, but that's my best guess as to why the histogram gains drastic peaks.
__________________
There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order.
foxyshadis is offline   Reply With Quote
Old 13th May 2018, 23:41   #6118  |  Link
Ma
Registered User
 
Join Date: Feb 2015
Posts: 324
I want to precise -- first command line gives o1.raw with full range inside, second gives o2.raw with limited range inside:
Code:
ffmpeg -i "../Color Range Test.MOV" -vf scale=out_range=pc,format=yuv420p -f rawvideo o1.raw
ffmpeg -i "../Color Range Test.MOV" -color_range pc -pix_fmt yuv420p -f rawvideo o2.raw
Is it ffmpeg bug or it should work that way?
Ma is offline   Reply With Quote
Old 14th May 2018, 01:18   #6119  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,814
Quote:
Originally Posted by Ma View Post
I want to precise -- first command line gives o1.raw with full range inside, second gives o2.raw with limited range inside:
Code:
ffmpeg -i "../Color Range Test.MOV" -vf scale=out_range=pc,format=yuv420p -f rawvideo o1.raw
ffmpeg -i "../Color Range Test.MOV" -color_range pc -pix_fmt yuv420p -f rawvideo o2.raw
Is it ffmpeg bug or it should work that way?
It's expected

The source is yuvj420p . The "j" means full range, and it's read correctly by ffmpeg

The order matters in ffmpeg. When you use -pix_fmt yuv420p, from a yuvj420p source - that tells ffmpeg to scale full range to limited range

If you were passing 8bit in/out , there is no need for any of it -f rawvideo is enough. It's just decoding to raw video. (But the receiving application might need you to enter parameters if piping raw)

In the earlier example, it was scaling to 10bit. That's why the -vf scale was used with format. (ffmpeg does not have a yuvj420p10le or be equivalent pixel format; the "j" is only for 8bit)
poisondeathray is offline   Reply With Quote
Old 14th May 2018, 06:22   #6120  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,814
I'm going to call this issue that Selur brought up, the 8bit=>10bit , 10bit encoding a bug. Both x264 , x265 are affected. You can see similar problem on other full range DSLR clips.

This is a test video from an old 7D review
http://www.imaging-resource.com/PRODS/E7D/E7DVIDEO.HTM

Direct Link
Lossless encoding doesn't exhibit the problem, nor does the scaling to 10 back to 8 to compare in vapoursynth only - so it's not a "tool chain" issue. But as low as crf 8 still clearly shows the coalescing values.

The likelihood that you'd get something like that is extremely low for several different videos (I tested 3 so far). You'd expect some quantization errors to round up, some down, but roughly be close to the original....especially at lower quantizers or crf values... Not redistribute to aggregate around 1 value, causing a line in the waveform. In this example, it was a higher midtone

The loss logic seems the same in x264 and x265, they both exhibit the same behaviour

lossless
https://s31.postimg.cc/av74s7963/lossless.png

x265_crf8
https://s31.postimg.cc/58aru7jcr/x265_crf8.png
poisondeathray is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 10:26.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.