Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 13th May 2018, 09:33   #6101  |  Link
Ma
Registered User
 
Join Date: Feb 2015
Posts: 326
After some tests it is a bug in ffmpeg (in my opinion).
Sorry for confusions.

Last edited by Ma; 13th May 2018 at 12:23.
Ma is offline   Reply With Quote
Old 13th May 2018, 09:42   #6102  |  Link
Jamaika
Registered User
 
Join Date: Jul 2015
Posts: 697
Quote:
Originally Posted by Ma View Post
If you use -pix_fmt yuv420p or yuv420p10 in ffmpeg it means limited range. Please use yuvj420 or yuv[B]
I know that I use x264, but...
Code:
[yuv4mpegpipe @ 000001d226440300] ERROR: yuv4mpeg can only handle yuv444p, yuv422p, yuv420p, yuv411p and gray8 pixel formats. And using 'strict -1' also yuv444p9, yuv422p9, yuv420p9, yuv444p10, yuv422p10, yuv420p10, yuv444p12, yuv422p12, yuv420p12, yuv444p14, yuv422p14, yuv420p14, yuv444p16, yuv422p16, yuv420p16, gray9, gray10, gray12 and gray16 pixel formats. Use -pix_fmt to select one.
Could not write header for output file #0 (incorrect codec parameters ?): I/O error
Error initializing output stream 0:0 --
or
Code:
[Parsed_format_1 @ 000001587211bec0] Invalid pixel format 'yuvj420p10'
[AVFilterGraph @ 00000158714fd980] Error initializing filter 'format' with args 'yuvj420p10'
Error reinitializing filters!
Failed to inject frame into filter network: Invalid argument
Error while processing the decoded data for stream #0:0
Jamaika is offline   Reply With Quote
Old 13th May 2018, 17:52   #6103  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,259
'yuvj420p10'
try:
Code:
-vf zscale=rangein=full:range=full -strict -1 -pix_fmt yuv420p10le
this seems to work for me when using ffmpeg als decoder, but that wouldn't help when using vspipe which poisondeathray tested,..
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 13th May 2018, 18:16   #6104  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,346
Quote:
Originally Posted by Selur View Post
'yuvj420p10'
try:
Code:
-vf zscale=rangein=full:range=full -strict -1 -pix_fmt yuv420p10le
this seems to work for me when using ffmpeg als decoder, but that wouldn't help when using vspipe which poisondeathray tested,..
But you still get problems unless you use lossless, right ?

You can look at histogram classic and there is a big line artifact in the waveform; you wouldn't expect that at low crf values (maybe at high lossy quantizers)

So it's not a pipe or toolchain problem
poisondeathray is offline   Reply With Quote
Old 13th May 2018, 19:04   #6105  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,259
Quote:
But you still get problems unless you use lossless, right ?
Not really. (used crf 18 and 8bit)

Uploaded my compare script and the files I use to my GoogleDrive
Argh,.. I used crf 18 and 8bit.
-> will try again with 10bit
You are right there are still 'line artifacts' in the histogram when using 10bit.


Cu Selur
Ps.: As a side note: either I'm making something wrong or x264 is totally broken when handling with pc scale video. (see: https://forum.doom9.org/showthread.php?p=1841998)
__________________
Hybrid here in the forum, homepage

Last edited by Selur; 13th May 2018 at 19:18.
Selur is offline   Reply With Quote
Old 13th May 2018, 19:17   #6106  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,346
In case it's a vpy or algorithm issue (you're using zimg/z.lib in vpy to convert back to 8bit to check at 8bit waveform) , I double checked with the vpy only also with nearest neighbor /Point instead of Bicubic - you don't get those artifacts (when going back/forth in the vpy only) ; you only get those artifacts with x264/x265 lossy encoding (and CRF 8 should be near lossless) . Lossless encoding fixes it, so it's definitely not a pipe or "toolchain" issue .

Last edited by poisondeathray; 13th May 2018 at 19:23.
poisondeathray is offline   Reply With Quote
Old 13th May 2018, 19:38   #6107  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,259
Quote:
Lossless encoding fixes it, so it's definitely not a pipe or "toolchain" issue .
Not necessarily, lossless should be YUV4:4:4 and not YUV4:2:0.
Atm. my best guess is that these line artifacts are due to the compression, but it might be a but.
__________________
Hybrid here in the forum, homepage

Last edited by Selur; 13th May 2018 at 19:43.
Selur is offline   Reply With Quote
Old 13th May 2018, 19:44   #6108  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,346
Quote:
Originally Posted by Selur View Post
Not necessarily, lossless should be YUV4:4:4 and not YUV4:2:0.
No; "lossless" is a relative term with respect to whatever your input is

If you have a YUV4:2:0 source, upscaling to YUV4:4:4 is not lossless (unless you use nearest neighbor to up and downscale back to YUV4:2:0)
poisondeathray is offline   Reply With Quote
Old 13th May 2018, 19:45   #6109  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,346
Quote:
Originally Posted by Selur View Post
Atm. my best guess is that these line artifacts are due to the compression, but it might be a but.

That's what it looks like (because lossless encoding fixes it), but it's very bizarre pattern (look at the classic histogram) . Many errors congregate around a low midtone value, creating a line artifact .
poisondeathray is offline   Reply With Quote
Old 13th May 2018, 19:56   #6110  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,259
okay, looking at the classic histogram view, I see what you mean, and I guess those might be due to compression artifacts.
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 13th May 2018, 20:02   #6111  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,346
You would expect random quantization artifacts to be more random, not coalesce around 1 value.

And you would expect it to be "less lossy" or higher quality, certainly at CRF 8 .

It might have just been a "perfect storm" conditions . But do you have another clip/scene/section from that camera to test ?
poisondeathray is offline   Reply With Quote
Old 13th May 2018, 20:13   #6112  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,259
Quote:
But do you have another clip/scene/section from that camera to test ?
Sadly, no. Got it from a Hybrid user who stumbled over the problem.
(wrote an email to the user whether he could share some additional footage from that source)
__________________
Hybrid here in the forum, homepage

Last edited by Selur; 13th May 2018 at 20:16.
Selur is offline   Reply With Quote
Old 13th May 2018, 22:14   #6113  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,558
Quote:
Originally Posted by Jamaika View Post
Never heard about it. A few threads further users advise against the use of rawvideo.

I test conveters:
Code:
ffmpeg_080518.exe -i "Color Range Test.MOV" -f yuv4mpegpipe -vf scale=1280:720:in_color_matrix=470bg:in_range=full:out_color_matrix=470bg:out_range=full,format=yuv420p10,lutyuv=val:val:val -strict -1 - | 
x264-10bit.exe --demuxer y4m --muxer mp4 --input-csp i420 --input-depth 10 --input-range pc --output-csp i420 --threads 4 --preset veryslow --tune grain --crf 28 --fps 29.970 --keyint 60 --nal-hrd none 
--colormatrix bt470bg --colorprim bt709 --transfer bt709 --range pc 
--output "x264_420p10le_crf28_1.mp4" -
Code:
x264-10bit_2851.exe --muxer mp4 --input-csp i420 --input-depth 8 --input-range pc --output-csp i420 --threads 4 --preset veryslow --tune grain --crf 28 --fps 29.970 --keyint 60 --nal-hrd none 
--colormatrix bt470bg --colorprim bt709 --transfer bt709 --range pc 
--output "x264_420p10le_crf28_2.mp4" "Color Range Test.MOV"
Code:
bpgenc_0.9.8.exe -v -b 10 -f 420 -c ycbcr_bt709 -e x265 -m 9 -q 28 -o x265_420p10le_crf28.bpg rgba.png
Result of the test:
1)bad range,
2)bad range, no latest version 10bit 2901 (2018),
3)good.

Conclusion:
For me, the problem is in BT470bg.
Rematrixing YUV in swscale goes through an RGB intermediate; it's not supposed to rematrix when you pass the same value in and out, but that's my best guess as to why the histogram gains drastic peaks.
foxyshadis is offline   Reply With Quote
Old 13th May 2018, 23:41   #6114  |  Link
Ma
Registered User
 
Join Date: Feb 2015
Posts: 326
I want to precise -- first command line gives o1.raw with full range inside, second gives o2.raw with limited range inside:
Code:
ffmpeg -i "../Color Range Test.MOV" -vf scale=out_range=pc,format=yuv420p -f rawvideo o1.raw
ffmpeg -i "../Color Range Test.MOV" -color_range pc -pix_fmt yuv420p -f rawvideo o2.raw
Is it ffmpeg bug or it should work that way?
Ma is offline   Reply With Quote
Old 14th May 2018, 01:18   #6115  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,346
Quote:
Originally Posted by Ma View Post
I want to precise -- first command line gives o1.raw with full range inside, second gives o2.raw with limited range inside:
Code:
ffmpeg -i "../Color Range Test.MOV" -vf scale=out_range=pc,format=yuv420p -f rawvideo o1.raw
ffmpeg -i "../Color Range Test.MOV" -color_range pc -pix_fmt yuv420p -f rawvideo o2.raw
Is it ffmpeg bug or it should work that way?
It's expected

The source is yuvj420p . The "j" means full range, and it's read correctly by ffmpeg

The order matters in ffmpeg. When you use -pix_fmt yuv420p, from a yuvj420p source - that tells ffmpeg to scale full range to limited range

If you were passing 8bit in/out , there is no need for any of it -f rawvideo is enough. It's just decoding to raw video. (But the receiving application might need you to enter parameters if piping raw)

In the earlier example, it was scaling to 10bit. That's why the -vf scale was used with format. (ffmpeg does not have a yuvj420p10le or be equivalent pixel format; the "j" is only for 8bit)
poisondeathray is offline   Reply With Quote
Old 14th May 2018, 06:22   #6116  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,346
I'm going to call this issue that Selur brought up, the 8bit=>10bit , 10bit encoding a bug. Both x264 , x265 are affected. You can see similar problem on other full range DSLR clips.

This is a test video from an old 7D review
http://www.imaging-resource.com/PRODS/E7D/E7DVIDEO.HTM

Direct Link
Lossless encoding doesn't exhibit the problem, nor does the scaling to 10 back to 8 to compare in vapoursynth only - so it's not a "tool chain" issue. But as low as crf 8 still clearly shows the coalescing values.

The likelihood that you'd get something like that is extremely low for several different videos (I tested 3 so far). You'd expect some quantization errors to round up, some down, but roughly be close to the original....especially at lower quantizers or crf values... Not redistribute to aggregate around 1 value, causing a line in the waveform. In this example, it was a higher midtone

The loss logic seems the same in x264 and x265, they both exhibit the same behaviour

lossless
https://s31.postimg.cc/av74s7963/lossless.png

x265_crf8
https://s31.postimg.cc/58aru7jcr/x265_crf8.png
poisondeathray is offline   Reply With Quote
Old 15th May 2018, 04:04   #6117  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,558
Quote:
Originally Posted by poisondeathray View Post
I'm going to call this issue that Selur brought up, the 8bit=>10bit , 10bit encoding a bug. Both x264 , x265 are affected. You can see similar problem on other full range DSLR clips.
Another possibility: It could be an artifact of the decoder, or even of the spec. I'd check against Mainconcept, or one of the GPU encoders & decoders. x264/x265 and ffmpeg/libav are so ubiquitous that it's hard to test anything else.
foxyshadis is offline   Reply With Quote
Old 15th May 2018, 15:53   #6118  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,346
Quote:
Originally Posted by foxyshadis View Post
Another possibility: It could be an artifact of the decoder, or even of the spec. I'd check against Mainconcept, or one of the GPU encoders & decoders. x264/x265 and ffmpeg/libav are so ubiquitous that it's hard to test anything else.

The testing routine used by selur converted back to 8bit to compare. A native 10bit waveform does not demonstrate the problem. But it's still a bizarre aggregation, and why this occurs at CRF 8 but not lossless.

I'll swap out and double check the decoders and encoders and do some more tests when I have more time
poisondeathray is offline   Reply With Quote
Old 18th May 2018, 13:52   #6119  |  Link
Barough
Registered User
 
Barough's Avatar
 
Join Date: Feb 2007
Location: Sweden
Posts: 480
x265 v2.7+351-d3b714bd5471 (GCC 7.3.0, 32 & 64-bit 8/10/12bit Multilib Windows Binaries)

Code:
https://bitbucket.org/multicoreware/x265/commits/all
Barough is offline   Reply With Quote
Old 21st May 2018, 12:25   #6120  |  Link
Ashok Kumar Mishra
Registered User
 
Join Date: Feb 2018
Posts: 13
x265 v2.8 released

Version 2.8
===========

Release date - 21/05/2018

New features
-------------
1. ption:`--asm avx512` used to enable AVX-512 in x265. Default disabled.
For 4K main10 high-quality encoding, we are seeing good gains; for other resolutions and presets, we don't recommend using this setting for now.

2. ption:`--dynamic-refine` dynamically switches between different inter refine levels. Default disabled.
It is recommended to use ption:`--refine-intra 4' with dynamic refinement for a better trade-off between encode efficiency and performance than using static refinement.

3. ption:`--single-sei`
Encode SEI messages in a single NAL unit instead of multiple NAL units. Default disabled.

4. ption:`--max-ausize-factor` controls the maximum AU size defined in HEVC specification.
It represents the percentage of maximum AU size used. Default is 1.

5. VMAF (Video Multi-Method Assessment Fusion)
Added VMAF support for objective quality measurement of a video sequence.
Enable cmake option ENABLE_LIBVMAF to report per frame and aggregate VMAF score. The frame level VMAF score does not include temporal scores.
This is supported only on linux for now.

Encoder enhancements
--------------------
1. Introduced refine-intra level 4 to improve quality.
2. Support for HLG-graded content and pic_struct in SEI message.

Bug Fixes
---------
1. Fix 32 bit build error (using CMAKE GUI) in Linux.
2. Fix 32 bit build error for asm primitives.
3. Fix build error on mac OS.
4. Fix VBV Lookahead in analysis load to achieve target bitrate.
Ashok Kumar Mishra is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 08:43.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.